# Dissecting rogue processes - Notes

## msalerno

I have read many posts on these forums where people find unknown/hacked processes wasting valuable resources.  I have also seen many replies to these posts on how to get more details about what the process is actually doing.  I think it would be beneficial if we were able to create a sticky topic that includes different methods of figuring out what the unknown process is doing.  Here is my attempt.

Taking into consideration the following post, I would have recommended the following steps:

1. Capture the output of 'lsof -n -P -p <pid of process> > ~/proflsof.log'

2. Record as much data as possible about the proc, your going to need a few screens

3. strace -p <pid of process> 

- probably a good idea to add '| tee ~/procstrace.log' to the strace 

- If possible, collect the strace data for a few minutes.

strace -p <pid of process>

- probably a good idea to add '| tee ~/procstrace.log' to the strace

echo CWD `readlink /proc/<pid of process>/cwd` > ~/procinfo.log

stat -L /proc/<pid of process>/exe >> ~/procinfo.log

cat /proc/<pid of process>/cmdline

cat /proc/<pid of process>/environ 

cat /proc/<pid of process>/

tar -cvf badproc.tar /proc/<pid of process>/

tar --dereference -cvf  ~/badexec.tar `readlink /proc/18729/exe`

4. Kill the process and move the files off of the box.

Files and dirs to evaluate in the badproc.tar:

proc/<pid of process>/cmdline

proc/<pid of process>/environ 

proc/<pid of process>/comm

proc/18729/fd/

I know that this is far from complete, but I'm hoping that others will contribute.Last edited by msalerno on Mon Aug 16, 2010 3:00 pm; edited 7 times in total

----------

## BradN

If the rogue process isn't smart enough to detect it, run it under strace and you'll have a very good idea what it's doing on your system and where it's connecting out to on the internet.

----------

## msalerno

I would think it would be best to do that in an isolated environment.

----------

## Hu

 *msalerno wrote:*   

> stat `readlink /proc/<pid of process>/exe` >> ~/procinfo.log

 No need to apply readlink here.  Use stat -L if you want to follow the link or plain stat if you want to capture the link properties.

 *msalerno wrote:*   

> I would think it would be best to do that in an isolated environment.

 You can attach strace to an existing process.  It may prove educational to collect a few minutes of strace data from the originally discovered rogue process before hitting the process with a SIGKILL.

----------

## msalerno

Agreed,  I updated the initial post with your recommendations.

Thanks

----------

## phajdan.jr

 *msalerno wrote:*   

> 4. Kill the process and move the files off of the box.

 

I think that after doing the analysis of memory contents (hopefully the malware doesn't detect that), you should try to preserve the disk contents before the malware can possibly revert its changes (power off the machine forcefully? there may be better options). Then you can make a backup of the disk image, and then do a further investigation of the disk. You have to reinstall the machine anyway.

----------

## BradN

 *Hu wrote:*   

> It may prove educational to collect a few minutes of strace data from the originally discovered rogue process before hitting the process with a SIGKILL.

 

That's what I was thinking - unless you just noticed the program _start_ running, chances are it's already compromised things, and it might help damage control to get a better idea what it did sooner than you would by disassembling the thing.

----------

## msalerno

True, but with the data collected from /proc, you could probably launch the app using the same env and args within an isolated environment.  To capture the strace from the time of execution.  Of course dependencies would be a pain unless you backed up all of the pids in /proc, even then it wouldn't get everything, but you could get an idea of what it was trying to do.

----------

## Hu

I do not think that anyone has suggested that rerunning it in an isolated environment is a bad idea.  However, there seems to be some confusion and/or disagreement about the merits of attaching strace to the already running exploit in the original environment upon first discovering that the system is compromised.  In my opinion, doing so has no downside and some potential upside.  There is no downside since the machine is already running hostile code, so there is no way to know in advance that the exploit has not already done all the damage it was meant to do.  Attaching strace to it might provoke it to become more destructive, but that would be a weird thing for an attacker to do.  There is some upside since you might learn something useful about the process from the strace.  You can always SIGKILL it when you are done tracing.

----------

## msalerno

I completely agree, launching the program with strace and attaching to an existing pd both can provide valuable information.  There is no debate there.

----------

## Hu

For the invocation of lsof, could you edit the initial post to suggest that it be run with -n -P?  This will suppress symbolic lookups, which both speeds up its execution and makes the output more deterministic.  When reverse resolution is done, connections to remote hosts may be shown with the host name, which can be ambiguous if the name resolves to more than one address.  Users can always run the reverse lookups later if they want to see the symbolic names.

----------

## msalerno

Good point, I updated the post.

----------

