Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
Privacy

Monitoring What Files Your Applications Leave Behind? 131

Posted by Cliff from the package-manifests-GOOOOOD! dept.
GoRK asks: "I have to install a commercial application on one of my servers. The application refuses to locate itself anywhere other than under the /usr tree, and I am concerned with it munging with my configuration under /etc as it automatically configures its daemons and whatnot. I am also a bit concerned with the method it goes about to verify its license on install. Is there any way I can run the installer in some sort of wrapper that allows me to monitor what files and network sockets get read/written to during the change so that I can monitor what data on my machine is getting moved around and also build a catalogue of every last little bit of the app in case I ever have to remove it?" In my opinion, a sensible software installer should have some form of user accessible package manifest included. Why should consumers trust third party software to "do the right thing" in the right locations, especially when installing software you don't have the source to.

"I am a big fan of MD5 sums and package management as a very reliable integrity check and I value the fact that every file on my system save user documents, etc. belongs to a package and I can verify its authenticity. I need to make sure that I know about each and every system file that gets modified during the installation. It would also be nice to see and control if it accesses anything under /dev/ just out of curiosity.

I've never been a diehard security freak before, but I just feel like it's 'time to do things right' so to speak. Is there a tool that will assist me here?"
This discussion has been archived. No new comments can be posted.

Monitoring What Files Your Applications Leave Behind? More

Monitoring What Files Your Applications Leave Behind?

Comments Filter:

  • If you were really concerned (Score:1)

    by Anonymous Coward
    You'd have named this mysterious "commercial application". After all, someone else reading slashdot may have installed it too, and know what it does.

    Of course, you could also ask the company that makes the software. I have a small hunch that they would also know.

    That presumes of course, that this software actually exists, and this isn't an Open Source Troll Story.

  • FAM (Score:1)

    by Anonymous Coward
    I dunno what OS you use, but on FreeBSD FAM is quite useful. You could combine it with a NIDS system.

  • Re:Simple. Geesh. (Score:1)

    by Anonymous Coward
    No.

    The guy who modded it down probably just has a clue as to how "locate" works. Something which is obviously lacking in yourself, and "Amp", despite his low UID.

    Note that he got modded up to 5 anyway, and the downmod was "overrated". Exactly the mod that "Amp" deserves. Ontopic, not a troll, nor flamebait, but a a factually incorect pile of drivel.

    Good to know there ARE some clueful moderators still. Bad to know that he was outvoted by four clueLESS idiots.

  • Re:chroot (Score:1)

    by Anonymous Coward
    Translation: I got me a program off of alt.binaries.warez.linux and need to make sure the installer is not contacting the company that owns it.

  • chroot (Score:4)

    by Anonymous Coward on Sunday May 27, 2001 @01:11PM (#194883)

    Run the application in a chroot'd sandbox.

  • If someone knows what to do when the install breaks some other piece of software, they know enough to use find to look for the breakage point.

    If they don't know how to use find, what use is an install log to them?

    I must say that if I were writing an installer for money, I might well keep a log (though not of all modifications to all the files) but I'd certainly focus most of my time on making it work right so that nobody needs the install log, whether or not they're competent to do anything with it.
  • In Debian:

    dpkg --get-selections > /usr/local/dpkg.selections

    When your drive goes to "Silicon Heaven"

    dpkg --set-selections /usr/local/dpkg.selections

    (Assuming you've already restored /usr/local from backup)
  • Grab protopkg [slackware.com] from the Slackware FTP site and write a prototype that wraps the install process. (No, protopkg doesn't care how the software is installed, or even that you're installing software... it just watches the system change and packs up whatever's new. How the system changes is left to you, the prototype author.)

    Or, (protopkg trick of the week, kids), write a prototype that just has "sleep 10" in the compile() function. When protopkg goes to sleep, hit ctrl-z to stop it, and do whatever you want to manually. Then when you're done, give that shell an 'fg' to let protopkg finish its work.

    Idunno about monitoring the network sockets... that's kinda weird.

  • Simple. Geesh. (Score:4)

    by Amphigory ( 2375 ) on Sunday May 27, 2001 @02:56PM (#194887) Homepage
    bash# locate /usr/ > /tmp/usrfiles.old

    do the install

    bash# locate /usr/ > /tmp/usrfiles.new

    bash# diff /tmp/usrfiles.old /tmp/usrfiles.new

    Or, if you wanted to get fancier and check for changed files:

    for i in `locate /usr/`
    do
    echo $i:`sum $i` >> /tmp/usrfiles.old
    done

    install

    for i in `locate /usr/`
    do
    if grep -v $i:`sum $i` /tmp/usrfiles.old; then
    echo $i >> /tmp/usrfiles.changed
    fi
    done

    ###

    Shell scripting is your friend. Learn it well.

    --

  • Linux strace doesn't trace fork(2)/clone(2) by default. You have to use -f for that. Read that man page, and practice by tracing a simple shell script to see what it does and what you can make of it. Use -s to show longer strings if you want. (The default is to show ... after 32 character strings, except for file names, which are always shown in full.)
    I recomment you strace all system calls from the installer (i.e. don't use -e), and filter later with grep | less, so you make sure you don't miss any interesting data.
    #define X(x,y) x##y
  • If you want a list of modified files, you could use checksumming utilities such as Tripwire [tripwire.com] or Aide [cs.tut.fi].

    If you want to see what filehandles are open - as in, files & sockets - lsof [purdue.edu] is useful.

    The checksummers take a couple of minutes to check timestamps, and at least a few minutes for checksumming; lsof could be scripted to run in a loop, I guess. These are tools for use at intervals. If you want to get a continual log, look at strace. If you want to be able to reverse the changes, you could try chroot, or back up your system, or use a test system.

  • Tripwire (Score:1)

    by GTM ( 4337 )
    This package is more security oriented, but Tripwire is the way to go, since you have no standard way of monitoring events on the file system under Un*x.

  • strace (Score:1)

    by khuber ( 5664 )
    you could run strace -o<file> <installer>
    on it, but you'll get a lot of detail to
    wade through unless you give more options

    you'd see all file accesses though

    -Kevin
  • should've mentioned that. most systems call
    what Linux's strace is truss

    -Kevin
  • The gripe isn't Windows per se, but
    that most Windows utilities seem single-minded
    unlike Unix utilities which can be used to
    do multiple things (because you can combine
    them in the shell), and also that Windows
    often requires third-party apps for some
    capabilities that are common on Unix systems.

    Neither of these issues have anything to do with
    the Windows OS itself (the kernel).

    -Kevin
  • 1) Yes, there is closed source software. (Somewhere on other peoples computers.)
    2) Yes, Open-Source Software has bugs just like any other software. But having the source enables you to fix it.
    3) Yes, you have to trust many people, if you use any kind of software. But it's easier to trust software engineers, who don't hide their source.
    --
    Raphael Wegmann
  • Alot of installers dont want you to know whats going on behind the scenes. They want to install time limited keys, so you get nagged about purchasing the products. If you could just rm the key, you basically bypass the time limit. This is where hacking 101 comes in, trussing a file, then using lsof and logging all files. Plus not to mention what files it alters, what about ports it opens?

    You should always play around with the application on a non-production machine before you go live.

    BTW, TripWire will catalog your entire system, and let you know if any new files are added or altered. Very good method for security also.

  • Well, you're using commercial software, right? You didn't just pay for a chunk of bits and bytes when you got it. Get your money's worth and call them to ask what changes the program will make to your system configuration. I think that's the easiest and simplest way.

  • VMware (Score:1)

    by Gerv ( 15179 )
    VMware [vmware.com] has cool stuff like non-permanent disks so you can install it, see what happened, and then roll your disk back to where it was before the install. Just what you need.

    Gerv

  • Hate to say it, but we do have that. It's called Tripwire. Been on Sourceforge since 1998, and I can recall mention of it back for years before that. Thing's a lifesaver when it comes to detecting anything messing with system directories.

  • True, but then on Unix all the configuration's on the filesystem anyway. Usually all you want in the end's a list of what changes the program made. Detailed step-by-step traces are useful only rarely, in debugging arcane problems. If I need more detail than Tripwire gives by default, I usually just diff the new files against a CD-RW backup from just before the install ( yes, I'm spoiled by being able to quickly back up my configuration that way ).

  • Not an answer to the original question, but do you guys know that OpenBSD policy is to never touch the /etc (neither from the ports, nor during the system upgrades)?

    As for logging the changed files, how about looking at mtree [openbsd.org] and tripwire?

  • You could run the install in a chroot and the package the install into a .deb file using the deb-make utilities and install that which would let you know if it is trying to overwrite any files that have already been installed and fail unless you send switches to force overwrites. If the original rpm or tgz simply needs to be unpacked to / to install you just need to alient packagename to change it to debian format. You can then get a list of the files installed by this package using dpkg -L packagename
    But if your using Redhat or other Linux distros using rpm formats i guess such an elegant solution
    is not available for you
  • The tools the previous poster mentioned actually
    intercept system calls to record exactly what
    a program did, tripwire only reports the
    difference between two snapshots of a filesystem.

  • Nothing against installwatch, but just for everyone's information, logwrites [yggdrasil.com] does much the same thing and has been around much longer. logwrites has been available under GPL since May, 1996 and was announced on comp.os.linux.announce on June 18, 1996.

    I think installwatch's distinct features are that it logs some operations that do not create or delete file names (like chown), has a handy shell script command and can use syslog. A distinct feature of logwrites is that it separates the system call traps from the logging into two libraries. The simple logging library can be used elsewhere for recording additional events relevent to installation history in the same format, without invoking the system call trapping. For example, we run modified cp, mv, and install to record where a file came from.

    Both programs are pretty trivial, so the duplication of effort has been tiny. Nevertheless, this situation illustrates that it can save you time to check around for an existing piece of free software before you start coding. You may be surprised what people have already released.

  • Yup. Minimum for any shop that considers itself professional:

    Production (it works)
    Test (we think it works)
    Development (we're trying to make it work)
    Training (the users are trying to figur out how it works)

    Our test platforms sometimes double as training platforms; nothing reveals incorrect assumptions quicker than a clueless user banging at an app.

    "What do you mean I shouldn't close while it says 'Committing transaction...'? If I shouldn't do it, why does it let me?"

  • TCT (Score:2)

    by ochinko ( 19311 )
    Or, The Coroner's Toolkit written by Dan Farmer (Earthlink) [fish.com] and Wietse Venema (IBM) [porcupine.org] will show you not only files that were written but also those that were deleted after the instalation and could even undelete them for you. Unfortunately, I don't think they have covered ReiserFS, but their kit works for many other filesystems, including ext2. Their kit was intended for post mortem examinations of compromised systems but should work in this case as well.

    You can also type:

    # find /etc -mmin 10
    to see all the files that were changed in /etc in the last ten minutes.

  • This program is fantastic. it will spy the system calls the your applacation will execute when it is installing. This way it knows exactly what files were "touched" and what werent even thougth the instalation did made any changes to a file (in this case MD5 would not detect).

    Simple do FM][ search on checkinstall and you will find it.

    --
    "take the red pill and you stay in wonderland and I'll show you how deep the rabbit hole goes"
  • The fakeroot package should do what you want:
    http://freshmeat.net/projects/fakeroot/
  • Using the install-root feature of RPM should go some distance, but to ensure that none of the scripts that might be run for pre- or post-installation do anything to you files you might additionally use chroot.

    Set up a minimal system in a directory of your choice. This might take some time, but if you tar the dir prior to doing the first installation you can recycle it later. the go to the dir and do

    chroot your-dir $SHELL

    and continue with installation as usual. should you be missing any file from the mini-system, just change to another shell and add.

    Afterwards you will not only know what your packages requires (never trust the Requires statement B-), but you can also be sure that no files outside of this directory have been changed.

    This solution has some advantages over the use-additional-system path, since your systems might be limited, especially if you require special hardware and stuff. Also a complete installation might take a while. taking out your trusty chroot-tar is much quicker.

    mfg 12decode

  • Isn't this why Microsoft has the "Designed for Windows" logo program, so that the consumer can have some form of belief that the program will install in a certain manner (ie. own DLLs in it's own directory, registry keys done properly).

    I wonder can there be a similar program made up for Linux? It would probably be fairly straight forward, with some complications as to where you prefer apps to install.
  • make -n install used to be really useful, but now most apps that use autoconf have an install target that iterates over several directories and output several hundred lines of junk. You really have to know what you're doing to find out what it's doing. I'd hate to have to use that feature to determine if something malicious was in there.

    For those kind of apps, it's faster to build a chroot (Debian chroots are really simple to make - unpack the base2_2.tgz to a dir somewhere, cd to it, and do chroot . bin/bash), make a copy of it, then do rsync --dry-run --verbose or diff -u --recursive on the two dirs to find out what changed.

  • Tripwire is designed as a security tool to tell you what files have been added, deleted and changed on your system but it sounds like it would easily the job you're looking for.

    You just run it once to generate a database of files on your system and again after installation to see what has changed. Easy!
  • The FHS specifies RPM 3.05 as the (current) standard for installing software on Linux systems.

    Where does the original poster say it's a Linux system?


  • Indeed! I should have heeded your advice earlier today. I tried to GRIP a CD, the application hung like a horse, so I tried ALT+CTRL+F2 to try and get a prompt. I got burried in output about DriveSeek and horror and error++, so I logged in as root and rebooted. Needless to say I spent the rest of the day recovering my HDD. Initially it didn't even show up on fdisk. Bugger.
  • Just as a quick side-note to the MS side of things... well actually Novell side.

    Novell has this funky "new" technology which basically goes like this:

    1) You Re-Image the box with your standard Base Image (ie. only OS + Drivers)
    2) You start the box (WinNT 4 sp5 for me) and login with a fresh account (ie. new profile)
    3) Run Novell's SNAPSHOT software (takes picture of hard drive - all files, reg, ini, etc)
    4) Install your app and configure.
    5) Run SNAPSHOT again (Which now takes another picture of your hard drive & configuration, then sits and compares them, throwing the differences to a file, and copying all changed files to a location you specified)
    6) Go for a drink (can take a while if its a big app)
    7) Import the differences file SNAPSHOT created into NDS, then review/change/mangle the application however you see fit.

    At the end of the day you've just created a "Application Object Template" which you can then import into Novell's NDS (where MS got its Active Directory idea :) ). Once you've tweaked the AOT (step 7 - and I do advise from experience that you at least review what changes the app makes - you'd be surprised how often things get "changed" that have *nothing* to do with what you want... IE setting changes comes to mind as a example), you can now push that application down to ANY NUMBER OF COMPUTERS.

    This product is called ZENworks, and it really is great - note, I didn't say it doesn't have problems and glitches. It comes in a few different flavors: Server, Desktop (and 1 other?)

    It's a really great idea/concept - lots of things currently available singlely, now nicely packaged into 1 thing. Unfortunately I haven't seen the equivalent in Linux/*nix yet ('course that *might* have something to do with *nix not yet having a nice Directory structure like NDS)
  • You are thinking of Reflections on Trusting Trust [acm.org], and it wasn't a compromise of gcc, it was just a "what if" situation that Ken Thompson talked about once.
  • I do believe I have herd at least on instance of someone introducing suspect code into gcc which caused login to be compiled with a backdoor password. I dont remember any web referances but I have seen it discussed in the professional Linux programming: published buy wrox search the index for security.
  • Use VMWare or Plex86. Install your application on the virtual machine, and there you can actually see your machine with all the changes. You can write the session to disk to try later, or just kill the VM if the app isn't what you want.

    Although I've personally never used one, there's dozens of interesing uses for VMs.
  • What makes you think that your clients can't change the microcode anyway, just because you don't give them source code?
    ------
  • Too bad the Linux companies won't use it.

    No, the InstallShield way of doing things is a POS that is the reason why Windows programs can trash DLL versions and such.

    Most of the modern unices have a central package management system that I would never trade for any per-package installation program. In fact, anyone could distribute a deb package, and all you'd have to do is run dpkg --force-depends -i commercialapp.deb && apt-get -f -y install, and the package and all its dependencies would get installed.
    ------

  • Does your Unix support DMAPI? Perhaps you could get a look at what's its doing using that. Maybe a plugin for ReiserFS, if you're using that, to get a look at what the changes are that it makes.

    - - - - -
  • Ah, why is this marked "informative"?

    I can't see even ONE piece of information that gives me information on ANYTHING!

    IMHO this should more likely be marked as "troll".

  • strace (Score:1)

    by stile ( 54877 )
    If you're really paranoid about any kind of file io, socket io, dev access, etc, run the installer under strace. It'll output many pages of stuff, but I imagine you can pipe it to a shell or perl script to get at the goodies you want.
  • InstallShield offers a Linux version now...has for awhile. Too bad the Linux companies won't use it. It is Java based, and it is pretty damned cool. And, of course, you can view files and thier install paths with the setup list. Would probably help in cases like this...

    http://www.installshield.com/iemp/specs/default.as p/a? [installshield.com]

  • strace is your friend (Score:3)

    by Convergence ( 64135 ) on Sunday May 27, 2001 @02:13PM (#194924) Homepage Journal
    strace, which uses posix tracing, can trace every system call made by an application.

    Actually, I frequently use it when debugging. Program doesn't start up correctly, I run it under strace to see if there's a file it can't find.

    It'll show you every subprocess, every kernel call, every file access, network access, etc.

    'strace netscape'

    or

    'strace -eopen netscape'

  • I have to install a commercial application on one of my servers. The application refuses to locate itself anywhere other than under the /usr tree,

    You didn't say whether the program was Open Source or proprietary, just that it was commercial. However, I'll assume that like most slashdotters you've never looked up the dictionary definition of commercial, or the Free Software Foundations confusing words list, and mean `closed source' instead. In reality a programs status as Free Software or Open Source or otherwise has no bearing on whether it is commercial or not.

    Either way, whether its pay per license proprietary software or Open / Free software that is produced for commercial reasons (meaning a support contract is avaliable), complain to your vendor.
    The FHS specifies RPM 3.05 as the (current) standard for installing software on Linux systems. nearly everyone who provides software on Linux (Open or proprietary) provides packages in this form, and the overwhelming majority of users use them. Get your moneys worth (either from licensing if its a closed source or support agreements otherwise) and tell the vendor you want packages.

    Where to put the app is another argument. I have no problem with a packaged app that wants to live in /usr. being `part of the OS' is fairly hard to determine on a platform without a standard distribution, and such arbitray information is useless to base a filesystem standard on.
  • and then you can restore whatever got wiped out from backups.

    You are using tripwire, right? And keeping good backups?

    *grin*
  • find .
    shows files & directories
    find . -name \*
    shows files only

  • Re:Simple. Geesh. (Score:3)

    by err666 ( 91660 ) on Sunday May 27, 2001 @04:24PM (#194928) Homepage
    locate only finds files that are already in its database. You'll need to use find instead for this to work.
  • $ touch /tmp/instdate.<package-name>
    $ <run the appropriate install procedure>
    $ find <tree where it installed itself, or if unknown, /> -newer /tmp/instdate.<package-name> -print

    Works for me... :)

    --
  • It's capable of detecting new files, changes to existing files and files that were deleted.

  • Tripwire? (Score:3)

    by tve ( 95573 ) <tripudium@ch[ ]o.nl ['ell' in gap]> on Sunday May 27, 2001 @01:15PM (#194931) Homepage
    Well, perhaps tripwire [tripwire.org] would be an option.

  • Re:This is where NT admins have it good :) (Score:3)

    by crucini ( 98210 ) on Sunday May 27, 2001 @11:19PM (#194932)
    Yes, the Windows crowd are like children or savages, easily dazzled by shiny objects (shrink-wrapped software) and perpetually dependent on parents/witch doctors (boxware vendors/MS) for the talismans to ward off a scary, complex world.
    But there are drawbacks to being an adult - we are burdened with the knowledge that the world is held together with duct tape and dried dung. For us there are no happy surprises in shiny packages, only the unerring certainty that software sucks and will continue to suck.
    With regard to shells, I agree - it's utterly amazing that Microsoft hasn't managed to "innovate" a decent shell yet. But I have used Bash on Windows, and it wasn't pretty. Windows is sluggish in many ways that come annoyingly to the fore in shell interaction.

  • Why don't you create a minimal chroot environment. Before you install you do a `ls -lR`, as well as after the install.

    Now just diff the ls-lR files and you see which files have been modified and which are newly created.

    (Just curious, isn't this something for a mailing list rather than an Ask Slashdot?)

  • If this is an RPM on a linux box use "--root somedir" to prefix things like /usr with "somdir". Other install methods like SVR4 "pkgadd" have similar relocate facilities.

    That would be --prefix somedir , and unfortunatelly it's not always an option, because some RPMs are not relocateable.

  • Umm, I guess, you're right.

    And that's all that I have to say about that. -- Forest Gump

  • Um...isn't 3

    3. ls -l `find ! -type d /` >> /files.1

    (Why the '\*'? Find defaults to '*' anyway. Seems like a waste of keystokes to me. *shrug*.)
  • everyone should have admin priveleges on their machine!
  • developers get admin rights over their machines :)
  • This is more-or-less my method. However a
    couple comments.

    I make a directory ( say, /root/changelog )
    and place all such package installation
    find-listings in that directory.

    The only other change is that most times,
    the installed files retain their original
    dates. To overcome this, I recommend using
    the find-flag -cnewer ( changed newer )
    sequence ( assume 'some-new-package' is the
    new software package )

    touch ./root/changelog/foo
    # install your software now
    find / -cnewer /root/changelog/foo -print \
    |tee /root/changelog/some-new-package-file.lst

    The file /root/changelog/some-new-package-file.lst
    will contain a list of all changed files on the
    system.

    hth

    -- kjh

    p.s. This is also useful for figuring out what
    files a GooEY admin menu affects.

    Do the touch ; Do the GooEY ; do the find-command
  • This is only peripherally related to the question, but I have some thoughts on how I would like package management systems in Unix to work.

    I want large tracts of the directory structure to be modified only by packages installing their standard files. To completely restore these areas, all you need know is what packages (and which versions) were installed.

    The ideal is something like this: I backup /etc, /home and /usr/local only. If my hard drive goes to Silicon Heaven, I buy a new one, restore onto it, and then run a program that looks at the list (saved in /etc) of packages I had installed and helps me reinstall them all.

  • Kind of offtopic, but is there anything like amiga's old "Snoopdos" for Linux?

    --

    "I'm surfin the dead zone
  • the chroot command was designed to do just this read the howto
  • man chroot
    try chrooting to a safe location first and see what it does. That will help verify that the script won't modify/have access to any files you don't want it to have access to.
    of course there's a good chance the script won't work either.

    A good firewall can monitor what network connections it starts up.

    -f

  • That's the beauty of make -n install which will just print the commands that make install will be running.

    Of course that won't work for install scripts that don't do make install's

    and for the original post...
    If you're *really* unsure about what the install scripts that the closed source programs you're installing will fuck up, then why are you trusting their actual app to run correctly?

    2 things-
    1- make sure you have good tech support from the company.
    2- install on a test server first, then on your production machine.

    -f

  • The /proc filesystem should tell you what files a process has open. Get the program's PID, then go to /proc/PID/fd and do an 'ls -l' to see what files (and sockets) the process has open. This will probably not give you all of the info you want, but it should be a good start. You can also try doing a 'find' on files modified since a minute before you ran the installer. I know that these are quick fixes, but, until you find something better...


    -------
  • Give this tool a shot, it may help, I used this with slackware in order to have _some_ kind of package managment for thrid party binarys/source builds: http://www.encap.org/ Good luck
  • It's not often there exists something in the NT world that doesn't have a parallel in *nix land, but this is one of 'em
    strace? It should come with all Linux regular linux distributions.

  • Re:chroot (Score:3)

    by BMazurek ( 137285 ) on Sunday May 27, 2001 @02:15PM (#194948)
    FreeBSD extends the ideas behind chroot with it's jail [freebsd.org] functionality.

    Using this, you could set up a virtual machine, similar to the way the Openroot Project [sektor7.ath.cx] does.

  • Offtopic? At least it gets a positive rating.

    I would have suggested the very same thing, for determining what files get installed/removed. Make a basic filesystem and chroot into it, and then install/run the commercial app. This way your system would be protected.

    Of course, my other suggestion would put things into a little more perspective:
    1. Make a complete system backup.
    2. as root: md5sum `find / -name \*` > /files.1
    3. ls -l `find / -name \*` >> /files.1
    4. install the commercial app.
    5. repeat steps 2 and 3, to different filenames(files.2).
    6. diff /files.1 /files.2

    the output of #6, if you were feeling properly bored, could be parsed by a perl or shell script if you wanted it pretty, otherwise you'd know what files were created, deleted, or modified.
    Of course, this solution wouldn't have the foresight to note the differences in other system changes, such as the log files... But you could deal with that yourself.
    btw, I assume bash as your shell... you can probably avoid the backslash escape before the asterisk in other shells. heh

  • mtree (Score:1)

    by mdray ( 146020 )

    The *BSDs have a tool called 'mtree' (in the base distro) which is like a cut-down version of tripwire. It's great for detecting trojaned binaries and seeing what files are touched by a program. It will check on timestamps, md5/sha1/chksum/other checksums, size, permissions, ownerships and more.

    cd /usr
    mtree -x -c -K md5digest > /tmp/before

    <run program>

    cd /usr
    mtree -f /tmp/before

    <mtree prints list of files that differ from before program ran>

    You can also run mtree with -K chksum to give you a checksum value for the whole directory (do this when you know the directory is 'clean' and write it down). Run mtree each night to generate a new checksum for the directory and then compare the two. If they're not the same, something in your directory tree has changed.

  • Install it normally, then see if it changed /etc. If it did, then just pull the plug and everything will be fine, thanks to our friend, ext2.
    ------------
  • Yeah--the pull-the-plug trick is a good one. One time, someone erased illegal files right in front of my eyes. I, of course, was trying to stop them. So once they had dropped the files in the Trash (this was a Mac) and went to Special->Empty Trash, I pulled the plug. They walked away, thinking they had foiled my plan.

    No way! I turned on the computer and found the files in their original location. Sometimes, asynchronous FSs can be helpful :-)
    ------------
  • If you want to trace what system calls are being made, have a look at strace, though if you want to make sure it doesn't actually modify anything, the idea of chrooting is probably a good one.
  • Many software installers will leave an installer log when they are done, but by then it can be too late. It seems rare that a software installer actually tells you what it is going to do before it does it. It is things like this that lead to unneeded tedium for the end user, such as backing up your configs before an install, 'just in case.' I just don't think we should be forced to go through this hassle, and I will make sure to clue the user in during/before the install in any software I write in the future. It had previously never occurred to me to do this, and I think it is the same way with most developers, be they commercial, open source, shareware, or whatever..
  • Every MCSE knows to install the software on a test system before installing it on a pruduction system. After you install it run a script that will compare the files before and after.

    Why try to start from scrach is the MS world has been exposed to this for years? Learn from other mistakes.
  • IMHO, what is needed here is a full versioning filesystem. Something like Katie [netcraft.com.au], written by Geoffrey Bennett, of Windows Refund fame... I believe it's still fairly primative, but perhaps if people like it, they could contribute :).

    Certainly, when something like this is available, production-ready, and fast enough, I would choose to use it on all my boxes...

  • Sounds to me like a textbook application for VMWare - if you have a box big enough, you can create a virtual machine and install you package there. If you take a snapshot before and after, comparisons are easy.

    Also, the VMWare wrapper will let you monitor anything that goes in or out the virtual machine's network connections, thus enabling you to identify the licence confirmation mechanism.

    I'll go back to lurking now.
    --
    If you want to live in a country ruled by religion, move to Iran.

  • Eh, even if there WAS SUCH TOOLS FOR UNIX, if you asked any good UNIX admin if you could install a tool on his/her system to do something as everyday and simple like this. He'd let out an evil laugh and run three commands that did the exact same thing as these tools before you could even download them. I always thought it was funny when I was a UNIX admin, and I worked along side some NT admins. They just think so in-the-box, I don't think it's thier fault though they don't even have an adequate shell! Things that I would have automated the hell out of they point/click and do manually every single day. Things that I could have written a simple 10 line shell script to do, they pay 1000 dollars for a site license for some 3rd party tool. It never ceased to amaze me...
  • Well yeah, tripwire. But really, you don't need something that elaborate. Just get a test box and:

    find / > file.list1 ; ./install ; find / > file.list2 ; diff file.list1 file.list2

    So maybe it's changing some files, and not only adding them. Turn off services that would be automatically changing files for a minute(you do all the services that are running on your system and what they are changing and when right?!). Install the app, check modification times for all the files on the system if they are later then the time you intalled the app, the app probably changed them. If I were admining a unix box, and someone suggested installing a FileMon, or SysDiff type tool on my system I'd laugh at them and tell them to write a script just for the point of the matter! I mean come on!
  • I've never tried it, but might running in a chroot'ed shell allow you to install it where you like, at least for a test run? Also you may be able to truss the pkgadd tool and log all of the system calls. Give it a shot with something you trust and see if it works out first. Don't forget to send a nastygram to the package author for not allowing configurability.

  • Why not just "find / > beforeinstall.txt" and "find / > afterinstall.txt" and use "diff" to compare them afterwards?

    You can use diff to check files in /etc or wherever if you have backups as well.....

  • I concur - I've been using installwatch [mayams.net] for quite a while, with excellent results; including with installers from commercial packages. Very nice utility. Use it as installwatch -o logfile [your installer name here].
  • Perhaps I should state that I run my own buisness, and admin my own servers. If it was sombody else's system, there is no way I would spend that much time on it. These servers are my pride and joy, and I work hard to make sure every one going up for production use is *PERFECT*. This means delving in to the kernel and changing things around, libc modifications, (re)compiling everything optomised for the particular CPU the machine has, etc.

    This course of action has really paid off. Every server I have put up has not needed any sort of modification/tweaking to this day. The machine goes up, and stays up, reliable as the hardware it is on.

    If I were for-hire, I would probably charge per-job, as my machines are very close to self maintaining. Once it's set up, a low-paid lackie could handle it.
  • Some people (noteably the best UNIX admins) take things like this VERY seriously, for a good reason. After putting 600 man-hours in to configuring a machine from the ground up, I know I wouldn't just go install somthing when I don't know what it will do to my system. Sure I have backups, but it should never come to that. In order to effectivly run a production server, you have to know how _everything_ installed on it works, and interacts with everything else. If you do not know, you waste lots of time with trial and error, and any modifications you make may cause other applications to misbehave, because you simply do not know how things interact. I agree 100% with the posters concern.
  • As part of a company that sells multi-million dollar hardware under warranty. We certainly DO NOT want someone who doesn't understand the hardware architecture, going in and 'tweeking' our proven microcode. Potentially damaging hardware and forcing *US* to pay for their incompetence.

    Come on now. That's just FUD. All warranties have certain things that will void them. For example, if you buy a new radio and it breaks you may be able to replace it, but if it broke because your 5 year old smashes it with a sledge hammer then you can't replace it because the warranty doesn't cover that.

    You simply say something in the warranty like "If you use drivers for this device that were not written by either <your company name> or a company or person who <your company name> has certified, then the warranty on this device will be void." And presto! You are not forced to pay to replace any hardware that someone uses a 3rd party driver on.

    Why open source the drivers then? So your consumers have more confidence in the quality, and they can fix bugs themselves and give them back to you.

    I realize that this contract prevents users from applying bug fixes that they wrote if they want the warranty, but I think it's a fair compromise. You give the bug fix back to the vendor and they can review it and roll it out in the next version of their driver. Probably won't be as fast as you'd like, but hey, the bug could have not been discovered and fixed at all.

    --
    Garett

  • You should listen to this. This is some of the best advice you're likely to get short of wasting your money on a "consultant." Where I work, we have three stages to production. In addition to the test box, we have a more tightly controlled "production test" environment and then changes get promoted to production.
    On the other hand, if you're a smaller company with limited resources and you happen to be running Linux, you may want to give serious consideration User-mode Linux [sourceforge.net]. From the site:

    "User-Mode Linux gives you a virtual machine that may have more hardware and software virtual resources than your actual, physical computer."

    I've played with it a bit and it gives you a complete (and completely sealed off) environment. It creates the entire environment within a file. So you could create the environment you want and then simply make duplicates using cp.
    It does require quite a bit of disk space since each VM is a complete system. So, if you want a virutal system with a 2GB filesystem you'll need +2GB of disk space, but heh, disk space is one of the cheaper components, certainly cheaper than a new system.
    Also keep in mind that it does "split up" your real system resources so you'll want to make sure you have plenty of RAM if you do any real work with it.
    Anyway, check it out and see if will help. Either way, you really do need to seperate test and new "stuff" from production.
  • You might find SUBTERFUGUE [subterfugue.org] interesting. It's kind of alpha, but it could do this, given the right trick.

    (Disclosure: I'm the principal author.)

    --Mike

  • So you're saying that commercial applications simply cannot use a sensible installation scheme, and must instead resort to hiding everything from the system administrator? So open-source really is superior, apparently.

  • Re:You're taking this too seriously (Score:5)

    by Grishnakh ( 216268 ) on Sunday May 27, 2001 @03:12PM (#194979)
    Oh please. You don't need to know how everything works. Microsoft made Windows easy to use and reliable, so all you have to do is use the install wizard and everything works great. If you're really curious about the inner workings, though, part of the way they achieved this unparalleled reliability and ease-of-use is through architectural decisions.
    1. The Registry. This piece of utter genious is crucial in an easily-maintained system. Instead of having a bunch of separate configuration files to keep track of, you just have one big binary file which all the install wizards access and change. Sure, sometimes one application screws it up for everything else, but that's no problem. Just re-install everything, no big deal. In this day and age, it's simply ridiculous to consider going back to ASCII text configuration files; it's just too difficult for certified system administrators to read these and edit them. Besides, we can always pass the blame to one of the third-party software vendors. I know that every time I have a computer problem at work and I need to meet a deadline, I can just tell my boss how some non-Microsoft program screwed up my whole system and he'll say "no problem, take as long you need. As long as we're using reliable MS software, we don't need to worry about deadlines."
    2. A central location for DLL's. By keeping all the DLL's in one or two directories, and allowing all applications to modify or overwrite them, we achieve breathtaking gains in system efficiency and reliability. Sure, sometimes you'll have minor problems with incompatible versions of the same DLL, but we can just blame that on the non-MS vendor and reinstall everything.

    Honestly, you talk like your time is precious or something. What's the big deal if you spend days setting up a system and some big-$$$ application screws it up? You just call their tech support (for $300+ per call) and tell them about your problem, then spend the next week or two redoing everything. And if it messes up that time, do it again. Using the intuitive and attractive Microsoft Windows user interface is such a pleasure, those hours (days, weeks...) will pass in no time. Plus, you can have Clippy (tm) help you! What other software company has such a fantastic, useful innovation?
  • what about doing the whole thing in a chroot environment and then comparing it with the "original" tree?
    Just my 0.02 euros..
    J.

  • installwatch - checkinstall (Score:5)

    by nsrbrake ( 233425 ) on Sunday May 27, 2001 @01:21PM (#194985) Homepage
    Checkinstall is a script that uses installwatch and rpm to build rpms from a source install. Do a search on freshmeat.net for checkinstall, the author also maintains installwatch. You only need to run installwatch as a wrapper to the install program and it will give you a listing of the files that it creates.

  • We NT folk have FileMon [sysinternals.com], RegMon [sysinternals.com], and Sysdiff/SMS Packager [microsoft.com] at our disposal.

    It's not often there exists something in the NT world that doesn't have a parallel in *nix land, but this is one of 'em
    ---
    nuclear presidential echelon assassination encryption virulent strain

  • Change your mount points (Score:3)

    by ConsumedByTV ( 243497 ) on Sunday May 27, 2001 @03:46PM (#194987) Homepage
    And then install it. Make a second /usr for the install. Or try using a small shell script that will traverse the file system and tell you all the recently modified files. Or just use a virtual machine.


    The Lottery:

  • Test it first! (Score:5)

    by JediTrainer ( 314273 ) on Sunday May 27, 2001 @01:31PM (#194996)
    I agree that it's probably a good idea to have a look at what it does (by running in some sort of sandbox), but here's my take on the whole situation in general...

    First off, you shouldn't be installing anything untested onto a production server. What your company should have is a box identical in configuration to your production box (or at least a development server. You DO have those, right?).

    Install the package on this server first. See what it touches. See if other packages misbehave after this is installed. Above all else, do not touch a production box period unless you've already seen what this program does.

    If your staging box blows up, so what? It's happened to me. Nobody really relies on it, and that's what the box is for anyway. No big deal. Document everything that happens when you install on this machine. Since it's exactly the same as your production box, if everything works, then all you have to do is follow your documentation to install onto your production server, and all will work fine.

    Trying to install stuff (even in sandboxes or wrappers) onto a server without testing it in a closed environment or staging area first is asking for trouble.
  • That's good in theory, but when you have application packages that install a wide variety of libraries and required portions of the program in different locations, or which go around and edit your configuration files, you ought to have some way of automatically cataloging what files were added, and what files were changed(and in what way) without having to fish through install scripts or hundreds of makefiles in a source package.

    This has been one of my biggest gripes in using anything with an autonomous installation mechanism. We all know that Windows machines can operate under the heavy load of junk and poorly formatted configuration files and dirty system registries. The case is not so simple with a Linux box, especially with a person like me who only has about 5 hours of spare time a day(only a fraction of which I ever really have free to use my computer). It's not quite enough time to go about and map all my .files and figure out which one a given package has skewed to the point where after I remove what files I know to have been installed into a package X breaks, or I can't use PPP anymore(both cases have happened).

    In the interest of fairness, I believe that in any case where someone is recieving money for some piece of software they have authored, there should be some way to remove the program without breaking the system.

    You can't ask a normal person to go about trying to find what configuration files a given package has disturbed, or what libraries it has upended or added. And installing development libraries causes whole new realms of problems.. But I think it's obvious where I'm going with this.

    Making sure anyone has the opportunity to use a stable operating system platform is the main reason I want to get into linux development.. :D

    TRiFIXION

  • While not wishing to fan these flames further, there are some (IMHO) good utilities from sysinternals [htp] which allow you to do this for free.

    filemon [sysinternals.com] will report file update/access and regmon will report registry update/access. [sysinternals.com]

    I haven't tried your examples, but I have found that even if the output from these is rather verbose, given some judicious regular expressions the output can be cut down to manageable size.

  • One Technique I have used... (Score:4)

    by IBitOBear ( 410965 ) on Sunday May 27, 2001 @01:16PM (#195004) Homepage Journal
    I used to keep a mock-up of a minimal system (or you can use a disposable separate computer) in a sub-directory and I would do test installs to that sub-directory using a chroot(ed) shell. This gave the opportunity to deconstruct things nicely. It was only used after other suspicous nonsense happened with a package or source.

    If this is an RPM on a linux box use "--root somedir" to prefix things like /usr with "somdir". Other install methods like SVR4 "pkgadd" have similar relocate facilities.

    If it is just a tar/cpio archive and an associated script to install, read the scripts.

    In short, there is no one tool to do this stuff, but it *is* doable by a number of means (on a *nix box) where it is otherwize impossible to do it on a windoze install-wizzard bundle.

    Hope this helps some.

    Rob.

    PS I have also had some luck by looking at the errors generated after trying to run the install as a normal, non priveliged, user.

    --
  • This is so damn beautiful [tear rolls down cheek]. You've succinctly captured what I've been bitching about for years in 3 sentences. I'm putting it on my door at work. Thanks, crucini.

    --

Slashdot Top Deals

He has not acquired a fortune; the fortune has acquired him. -- Bion

Close