# apache2 not starting on hardened

## turtles

Moving to a from a gentoo server to a newer gentoo hardened server today.

Had apache2 running.

Had sql-ledger running (DBI Perl)

```
/etc/init.d/apache2 start

 * Caching service dependencies ...                                                                                    [ ok ]

 * Starting apache2 ...

apache2: Could not determine the server's fully qualified domain name, using 127.0.0.1 for ServerName

(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80

no listening sockets available, shutting down

Unable to open logs                                                                                                    [ !! ]

```

Apache2 seemed to hang when I was messing around with confuguring cups. Maybe ssl related.

Not much in the error logs.

```
uname -a

Linux parsons 2.6.18-hardened #14 Mon Feb 12 22:52:08 PST 2007 i686 AMD Athlon(tm) XP 1800+ AuthenticAMD GNU/Linux

```

Any ideas?

Is it something to do with hardened?

I am new to hardened way of doing things.

EDIT: OK killing the pid it will restart.

Upon restarting it never loads anything.

OK does this seem normal??

```
netstat -plant

Active Internet connections (servers and established)

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name

tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      9773/apache2

tcp        0      0 0.0.0.0:3632            0.0.0.0:*               LISTEN      8738/distccd

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      8896/sshd

tcp        0      0 0.0.0.0:631             0.0.0.0:*               LISTEN      8637/cupsd

tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN      8838/postmaster

tcp       64      0 192.168.0.10:631        192.168.0.2:53514       CLOSE_WAIT  -

tcp       64      0 192.168.0.10:631        192.168.0.2:53513       CLOSE_WAIT  -

tcp       64      0 192.168.0.10:631        192.168.0.2:53506       CLOSE_WAIT  -

tcp       64      0 192.168.0.10:631        192.168.0.2:53504       CLOSE_WAIT  -

tcp       64      0 192.168.0.10:631        192.168.0.2:53508       CLOSE_WAIT  -

tcp       64      0 192.168.0.10:631        192.168.0.2:53509       CLOSE_WAIT  -

tcp      383      0 192.168.0.10:631        192.168.0.2:42312       CLOSE_WAIT  -

tcp      456      0 192.168.0.10:80         192.168.0.2:44686       CLOSE_WAIT  -

tcp      456      0 192.168.0.10:80         192.168.0.2:44697       CLOSE_WAIT  -

tcp      456      0 192.168.0.10:80         192.168.0.2:44699       CLOSE_WAIT  -

tcp     1030      0 192.168.0.10:54331      63.166.28.8:80          CLOSE_WAIT  8581/freshclam

tcp     1030      0 192.168.0.10:54332      63.166.28.8:80          CLOSE_WAIT  8581/freshclam

tcp      455      0 192.168.0.10:80         192.168.0.2:44700       ESTABLISHED -

tcp        0    144 192.168.0.10:22         192.168.0.2:51884       ESTABLISHED 9168/sshd: turtle [

tcp        1      0 192.168.0.10:631        192.168.0.2:53482       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53483       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53480       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53481       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53486       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53487       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53484       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53485       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53478       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53479       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53476       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53477       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53498       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53496       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53497       CLOSE_WAIT  8637/cupsd

tcp       64      0 192.168.0.10:631        192.168.0.2:53502       CLOSE_WAIT  -

tcp       64      0 192.168.0.10:631        192.168.0.2:53503       CLOSE_WAIT  -

tcp       91      0 192.168.0.10:631        192.168.0.2:53500       CLOSE_WAIT  8637/cupsd

tcp       64      0 192.168.0.10:631        192.168.0.2:53501       CLOSE_WAIT  -

tcp        1      0 192.168.0.10:631        192.168.0.2:53490       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53491       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53488       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53489       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53494       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53495       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53492       CLOSE_WAIT  8637/cupsd

tcp        1      0 192.168.0.10:631        192.168.0.2:53493       CLOSE_WAIT  8637/cupsd

parsons apache2 #                                                                                         
```

----------

## turtles

Well it has to do with: 

```
E [21/Mar/2007:20:02:26 -0700] Unable to open listen socket for address :::631 - Address family not supported by protocol.

E [21/Mar/2007:20:04:43 -0700] Unable to open listen socket for address :::631 - Address family not supported by protocol.

E [21/Mar/2007:20:21:08 -0700] Unable to open listen socket for address :::631 - Address family not supported by protocol.

E [22/Mar/2007:01:56:19 -0700] encrypt_client: Unable to encrypt connection from 192.168.0.2!

E [22/Mar/2007:01:56:19 -0700] encrypt_client: Unable to encrypt connection from 192.168.0.2!

E [22/Mar/2007:01:56:19 -0700] encrypt_client: Error in the push function.
```

I would like my connections to be encrypted but I guess I dont have it set up right.

----------

## deface

98)Address already in use: make_sock: could not bind to address 0.0.0.0:80 <-- you tried to restart too soon

E [21/Mar/2007:20:02:26 -0700] Unable to open listen socket for address :::631 - Address family not supported by protocol. <-- ipv6

apache2 & cups are 2 independent procs, neither interact with eachother. try looking at your /var/log/apache2/* logs, enable debug via

 -D INFO in your apache2 opts at /etc/conf.d/apache2

----------

## Hu

Which kernel features are you using: SELinux, RSBAC, or GRSecurity?  Are you trying to start Apache with the kernel's mandatory access control features enabled?  If so, check that you have configured the ACL correctly for Apache.  The error stating that it could not open logs suggests that either you have an incorrect path for logging or your ACL for Apache is denying it access to the logs it is supposed to write.

----------

## turtles

 *Hu wrote:*   

> Which kernel features are you using: SELinux, RSBAC, or GRSecurity?  Are you trying to start Apache with the kernel's mandatory access control features enabled?  If so, check that you have configured the ACL correctly for Apache.  The error stating that it could not open logs suggests that either you have an incorrect path for logging or your ACL for Apache is denying it access to the logs it is supposed to write.

 

Thanks for the tips folks!

Gentoo hardened and security in general is new to me.

netstat -plant looks normal after a reboot. I am concluding cups had crashed. I have done an update world and revdep-rebuild since.

MY kernel features are 

```

 [ ] Support soft mode                                                                         │ │

  │ │                       [*] Use legacy ELF header marking                                                             │ │

  │ │                       [*] Use ELF program header marking                                                            │ │

  │ │                           MAC system integration (none)  --->                                                       │ │

  │ │                                                                                                                     │ │

  │ │                                                       
```

```
──────────────────────────────────────────────────────────────────────────────────────────────┐ │

  │ │                       [*] Enforce non-executable pages                                                              │ │

  │ │                       [*]   Paging based non-executable pages                                                       │ │

  │ │                       [*]   Segmentation based non-executable pages                                                 │ │

  │ │                               Default non-executable page method (SEGMEXEC)  --->                                   │ │

  │ │                       [*] Emulate trampolines                                                                       │ │

  │ │                       [*] Restrict mprotect()                                                                       │ │

  │ │                       [ ]   Disallow ELF text relocations                                                           │ │

  │ │                       [ ] Enforce non-executable kernel pages                                                       │ │

  │ │                                                                                         
```

```
 [*] Address Space Layout Randomization                                                        │ │

  │ │                       [*]   Randomize kernel stack base                                                             │ │

  │ │                       [*]   Randomize user stack base                                                               │ │

  │ │                       [*]   Randomize mmap() base                                                                   │ │

  │ │                                                                                                                     │ │

  │ │                                                          
```

I have not done much more than click on a bunch of security stuff and followed a couple of guides.

I have checked out access control lists in the wiki and I am thinking not. 

The reason is because I dont give shell access to this server to more than 2 people. One is me and the other a programer whom needs access temporarily.

I did set a limit on the number of processes a user can use. Could that have affected apache or cups?

I cant remember where I set that stuff.

The server is only to run sql-ledger, postgresql, egroupware, apache2, cups.

log ins are only allowed via ssh.

----------

## Hu

Those look like options from the GRsecurity patch.  Is the system currently enforcing GRsecurity ACLs (enabled via gradm -E)?  If I recall correctly, GRsecurity kernels tend to be somewhat chatty about ACL violations, signals, etc.  Try to start Apache, then check your system logs to see if the kernel has printed any diagnostics about what Apache was doing.  If that does not reveal anything, you could try emerging dev-util/strace and having it follow the Apache startup, but that will probably generate a large volume of data.

I doubt Apache would exceed your process limits during startup, unless you set them very low.  I have an Apache running an almost-stock configuration, which only requires four processes when it is idle.  If you are concerned about limits, run ulimit -a -S; ulimit -a -H and post the output.  That will show the soft and hard limits that the shell is using.

Note that the ACL support you linked to is different from the ACL that may be in effect here.  The ACLs discussed on the Wiki are an extended form of discretionary access control, which let you grant extra access beyond the standard user/group/other model.  The ACL that may be in effect here is a form of mandatory access control, which is typically used to restrict access more than the standard model allows.  For instance, you might want a rule that says that Apache is only allowed to exec programs in /var/www/localhost/cgi-bin and nowhere else.  Such a rule would restrict the actions of a user who compromised Apache.  Well-written MACLs typically follow the principle of least privilege: the subject described in the MACL is permitted to do only those things which are necessary for it to perform its duties, and nothing else.  My concern is that Apache is being subjected to a MACL which grants it so little privilege that it cannot do some tasks it should do, such as write log files.

----------

## turtles

thanks for sticking with this Hu.

 *Hu wrote:*   

> Try to start Apache, then check your system logs to see if the kernel has printed any diagnostics about what Apache was doing.

 

All services started fine after the reboot but I do find this unusual message repeated about 100x in dmesg:

```
002 00000001 20364a9e ffffffff 00000000 593f3190 20261325 00000001 2050fe48 593f31a8

PAX: execution attempt in: <anonymous mapping>, 475cd000-475f5000 475cd000

PAX: terminating task: /opt/blackdown-jdk-1.4.2.03/bin/javac(javac):19306, uid/euid: 0/0, PC: 475cd040, SP: 5b67d4dc

PAX: bytes at PC: 55 8b ec 53 9c 58 50 8b c8 33 d2 81 f0 00 00 04 00 50 9d 9c

PAX: bytes at SP-4: 0000001c 4b9ad4cb 5b67d528 5b67d52c 0000001c 4b9efe48 00000001 ffffffff 5b67d568 4b742c83 080ef840 00000002 00000001 4b844a9e ffffffff 00000000 5b67d530 4b741325 00000001 4b9efe48 5b67d548

PAX: execution attempt in: <anonymous mapping>, 24198000-241c0000 24198000

PAX: terminating task: /opt/blackdown-jdk-1.4.2.03/bin/javac(javac):21343, uid/euid: 0/0, PC: 24198040, SP: 5846bbdc

PAX: bytes at PC: 55 8b ec 53 9c 58 50 8b c8 33 d2 81 f0 00 00 04 00 50 9d 9c

PAX: bytes at SP-4: 0000001c 21ff64cb 5846bc28 5846bc2c 0000001c 22038e48 00000001 ffffffff 5846bc68 21d8bc83 080edce8 00000002 00000001 21e8da9e ffffffff 00000000 5846bc30 21d8a325 00000001 22038e48 5846bc48

atkbd.c: Keyboard on isa0060/serio0 reports too many keys pressed.

```

Note the keybord error only appears once and was probably due to a cat trying to log in by sleeping on the keybord.

```
ulimit -a -S; ulimit -a -H

core file size          (blocks, -c) 0

data seg size           (kbytes, -d) unlimited

max nice                        (-e) 0

file size               (blocks, -f) unlimited

pending signals                 (-i) 6143

max locked memory       (kbytes, -l) 32

max memory size         (kbytes, -m) unlimited

open files                      (-n) 1024

pipe size            (512 bytes, -p) 8

POSIX message queues     (bytes, -q) 819200

max rt priority                 (-r) 0

stack size              (kbytes, -s) 8192

cpu time               (seconds, -t) unlimited

max user processes              (-u) 6143

virtual memory          (kbytes, -v) unlimited

file locks                      (-x) unlimited

core file size          (blocks, -c) unlimited

data seg size           (kbytes, -d) unlimited

max nice                        (-e) 0

file size               (blocks, -f) unlimited

pending signals                 (-i) 6143

max locked memory       (kbytes, -l) 32

max memory size         (kbytes, -m) unlimited

open files                      (-n) 1024

pipe size            (512 bytes, -p) 8

POSIX message queues     (bytes, -q) 819200

max rt priority                 (-r) 0

stack size              (kbytes, -s) unlimited

cpu time               (seconds, -t) unlimited

max user processes              (-u) 6143

virtual memory          (kbytes, -v) unlimited

file locks                      (-x) unlimited

```

----------

## chvo

Well, the PAX error message is problably explained by the use of java:

http://www.gentoo.org/proj/en/hardened/hardenedfaq.xml?style=printable#paxjava

use chpax to correct this (as explained on the FAQ page: you need to disable the necessary protections, since PaX will be upset by the code java generates).

----------

## turtles

Hey thanks.

Strange I wonder what is trying to use java?

Sql-ledger is written in Perl. Maybe cups?

----------

## chvo

 *Quote:*   

> Strange I wonder what is trying to use java?

 

Maybe your programmer colleague?    :Smile: 

It seems that some versions of postgresql in portage have a java USE flag. As far as I can see, CUPS has nothing to do with it.

----------

## Hu

 *turtles wrote:*   

> thanks for sticking with this Hu.
> 
>  *Hu wrote:*   Try to start Apache, then check your system logs to see if the kernel has printed any diagnostics about what Apache was doing. 
> 
> All services started fine after the reboot but I do find this unusual message repeated about 100x in dmesg:

 

Am I correct that Apache works when it is started through the system's boot process, but if you attempt to restart it from a root shell, then it fails?

 *turtles wrote:*   

> 
> 
> ```
> ulimit -a -S; ulimit -a -H
> 
> ...

 

Those limits all look fine.  It may be time to use strace to find what is going wrong.  Run it as strace -f -tt -o /tmp/apache-strace /etc/init.d/apache2 start.  See the strace manpage for a full list of options.  When it completes, open /tmp/apache-strace in your favorite text editor.  It will likely be far too long to post here.  You may be able to put it on one of the "paste bin" sites that some IRC users favor.  Otherwise, you'll be on your own to find the cause.  I would suggest first looking for calls which fail with EACCES, as well as any calls pertaining to the Apache log files.  Feel free to post snippets here for group review if you see something you think is not right.

----------

## turtles

 *Hu wrote:*   

> 
> 
> Am I correct that Apache works when it is started through the system's boot process, but if you attempt to restart it from a root shell, then it fails?

 

That was the problem at first. Something crashed cups hard the first time. As all its processes had ? after them.

If I log in as root an do an /etc/init.d/apatche2 restart it restarts fine. 

However I still cant get the web application Sql-ledger to play nice with cups.

When ever I try to print the kernel kills the javaapplet.

Printing in SQL-Ledger works by passing STDOUT to lpr. It looks like STDOUT passed to lpr triggers a java applet. I guess cups creates the applet? The programer and the developer of sql-ledger have confirmed they are not calling the applet.

I guess the question now is what is the best way to allow the javaapplet for printing and not cause a gaping security hole.

I am looking into that wiki.

Thanks all.

----------

## turtles

A possible soloution to this is to use the -P flag with the lpr command that sql-ledger uses. I have switched back to regular Gentoo in the mean time and have not tested this.

----------

