# Virtual Hosting Project

## JoeG

Hey folks.  Lemme just say that Gentoo is absolutely the best distro I have used yet, bar none.  Easy to set up, forces you to understand the system, easy to keep up to date.  But the forums community is what has kept me with it.  Absolutely the best community anywhere.  Which brings me to my question:

We have a Virtual Mailhosting Howto.  This one only sets up virtual mailhosts (as the name implies...and uses apache and phpmyadmin to configure them), but not virtual web sites.  We have a Virtual Hosting Made Easy discussion, but it doesn't really address hosting email for virtual domains.

  Kudos to  the authors of both, but if someone wants to set up a complete hosting service, they need it all.  Is there now, or is there any project in the works that does it all  :Question:   If anyone (or group) is working on this currently for gentoo, I'd be more than happy to help out.  I do have limited experience with this, but I learn fast, and am not above handling trivial tasks and let more experienced folks code the hard stuff.  Any takers?

JoeG

----------

## JoeG

nobody?    :Confused: 

Joe

----------

## JoeG

Well,  I've been hammering away at it, and have it down to two scripts and some support files after you have followed the Virtual Mailhosting Guide.    :Very Happy:  I'll post the text of the scripts as well as the directory structure for my solution next time.

Happy Hosting

JoeG

----------

## JoeG

Here they are.  You will need to have followed the Virtual Mailhosting Howto.  All scripts must be run from the scripts directory.  You will need to create the following directory structure:

```
/root

   |

   |______/scripts

                  |_________hostedmailuser

                  |

                  |_________hostedsite

                  |

                  |_________mkhostedskel

                  |

                  |_________README

                  |

                  |________/support_files

                           |

                           |________config.php

                           |

                           |________vhosts
```

The hostedmailuser script:

```
#!/bin/bash

#

#Setup script for virtual users email directories

#

#Static variables

VMDIR=/home/vmail

SPTDIR=./support_files

echo "For what domain are you creating users"

read DOMAIN

echo "What user account to add?"

read USER

echo "What is the user's first name?"

read FNAME

echo "What is the user's last name?"

read LNAME

echo "What is the password to use?"

read PASSWORD

#Quick sanity check to ensure that the hosted domain

#home folder doesn't already exist

if [ -d /home/vmail/$DOMAIN ]

   then

      echo "$DOMAIN directory already exists"

      echo "Creating home directory for user"

   else

      mkdir /home/vmail/$DOMAIN

fi

mkdir /home/vmail/$DOMAIN/$USER

maildirmake /home/vmail/$DOMAIN/$USER/.maildir

chmod 0700 -R /home/vmail/$DOMAIN/$USER

chown -R vmail:vmail /home/vmail/$DOMAIN/$USER

#Now we need to add the user account information to the SQL

#Database so that it is ready for use

echo -e "\t$USER@$DOMAIN\t$PASSWORD\t$FNAME $LNAME\t1009\t1009\t$VMDIR\t$VMDIR/$DOMAIN/$USER/.maildir/\t\ty" > $SPTDIR/$USER.$DOMAIN.sql

echo -e "\t$DOMAIN\tvirtual:" > $SPTDIR/trans.$DOMAIN.sql

cp $SPTDIR/$USER.$DOMAIN.sql /tmp/

cp $SPTDIR/trans.$DOMAIN.sql /tmp/

echo "LOAD DATA LOCAL INFILE '/tmp/$USER.$DOMAIN.sql' INTO TABLE users;" > batchsql.txt

echo "LOAD DATA LOCAL INFILE '/tmp/trans.$DOMAIN.sql' INTO TABLE transport;" >> batchsql.txt

echo "You will now be prompted to enter the SQL database password."

mysql -u mailsql -p mailsql < batchsql.txt

#Clean up the /tmp directory

rm -f /tmp/*.sql

echo "User creation complete for $USER@$DOMAIN".

```

The hostedsite script:

```
#!/bin/bash

#

#Setup script for virtual website users and directories

#

#We need to create the virtual site's directory structure

#and create a single user account that can ftp into that 

#site to maintain, while not allowing shell access

echo "What hosted site are you creating today?"

read DOMAIN

echo "What user name will control this site?"

echo "This user will be able to use ftp to transfer files"

echo "to the hosted site.  It does not have to be the same as"

echo "one of the email addresses for the domain."

read USER

echo "What is the password to use?"

read PASSWORD

#Sanity check to make sure that the site directory is not already there

#If not,  then gogogo.

if [ -d /var/www/$DOMAIN ]

   then

      echo "Sorry, that site directory exists"

      exit 1

   else

      mkdir /var/www/$DOMAIN

      cp -a /etc/skel-www/* /var/www/$DOMAIN

      perl -pi.bak -e "s/virthost/$DOMAIN/" ./support_files/config.php

      cp ./support_files/config.php /var/www/$DOMAIN/htdocs/squirrelmail/config/

      chmod 0775 -R /var/www/$DOMAIN

      useradd $USER -g apache -d /var/www/$DOMAIN -p $PASSWORD -s /bin/bash

      chown -R $USER:apache /var/www/$DOMAIN

   #Now we need to create the virtual site information file for Apache,

   #integrate it into the server, and restart the service.  The resulting

   #config file will be stored as ./support_files/$DOMAIN.vhosts

      perl -pi.bak -e "s/virthost/$DOMAIN/" ./support_files/vhosts

      cat ./support_files/vhosts >> /etc/apache2/conf/vhosts/vhosts.conf

      /etc/init.d/apache2 restart

   ###Put everything back like we found it

      cp ./support_files/vhosts ./support_files/$DOMAIN.vhosts

      mv ./support_files/vhosts.bak ./support_files/vhosts

      cp ./support_files/config.php ./support_files/$DOMAIN.config.php

      mv ./support_files/config.php.bak ./support_files/config.php

      echo "Virtual Site and User creation completed for $DOMAIN."

      echo " www.$DOMAIN should now be available.".

fi

```

The mkhostedskel script:

```
#/bin/bash

#

#We need a skeleton directory for future hosted sites

#

#Sanity check...we don't want to overwrite stuff

if [ -d /etc/skel-www ]

   then

      echo "Directory exists"

   else

      echo "Creating directory"

      mkdir /etc/skel-www

      cp -a /var/www/localhost/* /etc/skel-www/

      echo "Skeleton structure created"

fi

```

The README file:

```
NOTE:  All scripts _must_ be run from this directory!

To create new email sites and users for hosted domains, run:

   sh hostedmailuser

To create a skeleton directory for future hosted web sites, run:

   sh mkhostedskel

To create new web sites and control users for hosted domains, run:

   sh hostedsite

```

The config.php file:  (This is just a modified copy of the original from squirrelmail's directory)

```
<?php

/**

 * SquirrelMail Configuration File

 * Created using the configure script, conf.pl

 */

global $version;

$config_version = '1.4.0';

$config_use_color = 2;

$org_name      = "virthost";

$org_logo      = SM_PATH . 'images/sm_logo.png';

$org_logo_width  = '308';

$org_logo_height = '111';

$org_title     = "SquirrelMail $version";

$signout_page  = '';

$frame_top     = '_top';

$provider_uri     = 'http://www.squirrelmail.org/';

$provider_name     = 'SquirrelMail';

$motd = "";

$squirrelmail_default_language = 'en_US';

$domain                 = 'virthost';

$imapServerAddress      = 'localhost';

$imapPort               = 143;

$useSendmail            = false;

$smtpServerAddress      = 'localhost';

$smtpPort               = 25;

$sendmail_path          = '/usr/sbin/sendmail';

$pop_before_smtp        = false;

$imap_server_type       = 'other';

$invert_time            = false;

$optional_delimiter     = 'detect';

$default_folder_prefix          = '';

$trash_folder                   = 'INBOX.Trash';

$sent_folder                    = 'INBOX.Sent';

$draft_folder                   = 'INBOX.Drafts';

$default_move_to_trash          = true;

$default_move_to_sent           = true;

$default_save_as_draft          = true;

$show_prefix_option             = false;

$list_special_folders_first     = true;

$use_special_folder_color       = true;

$auto_expunge                   = true;

$default_sub_of_inbox           = true;

$show_contain_subfolders_option = false;

$default_unseen_notify          = 2;

$default_unseen_type            = 1;

$auto_create_special            = true;

$delete_folder                  = false;

$noselect_fix_enable            = false;

$default_charset          = 'iso-8859-1';

$data_dir                 = SM_PATH . 'data/';

$attachment_dir           = $data_dir;

$dir_hash_level           = 0;

$default_left_size        = '150';

$force_username_lowercase = false;

$default_use_priority     = true;

$hide_sm_attributions     = false;

$default_use_mdn          = true;

$edit_identity            = true;

$edit_name                = true;

$allow_thread_sort        = false;

$allow_server_sort        = false;

$allow_charset_search     = true;

$uid_support              = true;

$theme_css = '';

$theme_default = 0;

$theme[0]['PATH'] = SM_PATH . 'themes/default_theme.php';

$theme[0]['NAME'] = 'Default';

$theme[1]['PATH'] = SM_PATH . 'themes/plain_blue_theme.php';

$theme[1]['NAME'] = 'Plain Blue';

$theme[2]['PATH'] = SM_PATH . 'themes/sandstorm_theme.php';

$theme[2]['NAME'] = 'Sand Storm';

$theme[3]['PATH'] = SM_PATH . 'themes/deepocean_theme.php';

$theme[3]['NAME'] = 'Deep Ocean';

$theme[4]['PATH'] = SM_PATH . 'themes/slashdot_theme.php';

$theme[4]['NAME'] = 'Slashdot';

$theme[5]['PATH'] = SM_PATH . 'themes/purple_theme.php';

$theme[5]['NAME'] = 'Purple';

$theme[6]['PATH'] = SM_PATH . 'themes/forest_theme.php';

$theme[6]['NAME'] = 'Forest';

$theme[7]['PATH'] = SM_PATH . 'themes/ice_theme.php';

$theme[7]['NAME'] = 'Ice';

$theme[8]['PATH'] = SM_PATH . 'themes/seaspray_theme.php';

$theme[8]['NAME'] = 'Sea Spray';

$theme[9]['PATH'] = SM_PATH . 'themes/bluesteel_theme.php';

$theme[9]['NAME'] = 'Blue Steel';

$theme[10]['PATH'] = SM_PATH . 'themes/dark_grey_theme.php';

$theme[10]['NAME'] = 'Dark Grey';

$theme[11]['PATH'] = SM_PATH . 'themes/high_contrast_theme.php';

$theme[11]['NAME'] = 'High Contrast';

$theme[12]['PATH'] = SM_PATH . 'themes/black_bean_burrito_theme.php';

$theme[12]['NAME'] = 'Black Bean Burrito';

$theme[13]['PATH'] = SM_PATH . 'themes/servery_theme.php';

$theme[13]['NAME'] = 'Servery';

$theme[14]['PATH'] = SM_PATH . 'themes/maize_theme.php';

$theme[14]['NAME'] = 'Maize';

$theme[15]['PATH'] = SM_PATH . 'themes/bluesnews_theme.php';

$theme[15]['NAME'] = 'BluesNews';

$theme[16]['PATH'] = SM_PATH . 'themes/deepocean2_theme.php';

$theme[16]['NAME'] = 'Deep Ocean 2';

$theme[17]['PATH'] = SM_PATH . 'themes/blue_grey_theme.php';

$theme[17]['NAME'] = 'Blue Grey';

$theme[18]['PATH'] = SM_PATH . 'themes/dompie_theme.php';

$theme[18]['NAME'] = 'Dompie';

$theme[19]['PATH'] = SM_PATH . 'themes/methodical_theme.php';

$theme[19]['NAME'] = 'Methodical';

$theme[20]['PATH'] = SM_PATH . 'themes/greenhouse_effect.php';

$theme[20]['NAME'] = 'Greenhouse Effect (Changes)';

$theme[21]['PATH'] = SM_PATH . 'themes/in_the_pink.php';

$theme[21]['NAME'] = 'In The Pink (Changes)';

$theme[22]['PATH'] = SM_PATH . 'themes/kind_of_blue.php';

$theme[22]['NAME'] = 'Kind of Blue (Changes)';

$theme[23]['PATH'] = SM_PATH . 'themes/monostochastic.php';

$theme[23]['NAME'] = 'Monostochastic (Changes)';

$theme[24]['PATH'] = SM_PATH . 'themes/shades_of_grey.php';

$theme[24]['NAME'] = 'Shades of Grey (Changes)';

$theme[25]['PATH'] = SM_PATH . 'themes/spice_of_life.php';

$theme[25]['NAME'] = 'Spice of Life (Changes)';

$theme[26]['PATH'] = SM_PATH . 'themes/spice_of_life_lite.php';

$theme[26]['NAME'] = 'Spice of Life - Lite (Changes)';

$theme[27]['PATH'] = SM_PATH . 'themes/spice_of_life_dark.php';

$theme[27]['NAME'] = 'Spice of Life - Dark (Changes)';

$theme[28]['PATH'] = SM_PATH . 'themes/christmas.php';

$theme[28]['NAME'] = 'Holiday - Christmas';

$theme[29]['PATH'] = SM_PATH . 'themes/darkness.php';

$theme[29]['NAME'] = 'Darkness (Changes)';

$theme[30]['PATH'] = SM_PATH . 'themes/random.php';

$theme[30]['NAME'] = 'Random (Changes every login)';

$theme[31]['PATH'] = SM_PATH . 'themes/midnight.php';

$theme[31]['NAME'] = 'Midnight';

$theme[32]['PATH'] = SM_PATH . 'themes/alien_glow.php';

$theme[32]['NAME'] = 'Alien Glow';

$theme[33]['PATH'] = SM_PATH . 'themes/dark_green.php';

$theme[33]['NAME'] = 'Dark Green';

$theme[34]['PATH'] = SM_PATH . 'themes/penguin.php';

$theme[34]['NAME'] = 'Penguin';

$theme[35]['PATH'] = SM_PATH . 'themes/minimal_bw.php';

$theme[35]['NAME'] = 'Minimal BW';

$default_use_javascript_addr_book = false;

$addrbook_dsn = '';

$addrbook_table = 'address';

$prefs_dsn = '';

$prefs_table = 'userprefs';

$prefs_user_field = 'user';

$prefs_key_field = 'prefkey';

$prefs_val_field = 'prefval';

$no_list_for_subscribe = false;

$smtp_auth_mech = 'none';

$imap_auth_mech = 'login';

$use_imap_tls = false;

$use_smtp_tls = false;

$session_name = 'SQMSESSID';

@include SM_PATH . 'config/config_local.php';

/**

 * Make sure there are no characters after the PHP closing

 * tag below (including newline characters and whitespace).

 * Otherwise, that character will cause the headers to be

 * sent and regular output to begin, which will majorly screw

 * things up when we try to send more headers later.

 */

?>

```

The vhosts file:

```
<VirtualHost *:80>

ServerName www.virthost

DocumentRoot /var/www/virthost/htdocs

<Directory "/var/www/virthost/htdocs">

Options Indexes FollowSymLinks

AllowOverride None

Order allow,deny

Allow from all

</Directory>

</VirtualHost>

#

```

I'm pretty new at shell scripting, but these do work.    :Wink:  Anyone interested in setting this up in PHP or TCL/TK for a graphical interface?

JoeG

----------

## Wilhelm

You started up a kewl thread btw. I myself am setting up hosting and have starting to re-script all snippets to my likings since i don't want to be manually doing all the shit myself.

I'm halfway through writing my BIND DNS templating script which allows you to add domains like this 

dns <add/del> domain.com   // Adds deletes the domain zone

dns list                                 // List the domains

Anyway i'm doing it using a step by step approach.

I'll be looking into your scripts aswell.

My setup will be the following

ext3 with quota support

apache2

mysql

mod_php

bind

postfix

courier-imap <-- if needed

squirrelmail

----------

## Wilhelm

btw

Isn't it possible to use one squirrelmail for all users as any other service provider would do?!?

----------

## JoeG

 *Quote:*   

> Isn't it possible to use one squirrelmail for all users as any other service provider would do?!?

 

Absolutely.  They'd just log in on the main site (not the hosted one) and use username@virtualdomain as the username.  My solution just allows for a more personalized feel for each hosted domain.  :Cool: 

----------

## JoeG

 *Wilhelm wrote:*   

> You started up a kewl thread btw. 

 

Thank you very much.    :Very Happy: 

 *Wilhelm wrote:*   

> courier-imap <-- if needed

 

You'll need to, if you want the email deliverable to the users.  Postfix is just the smtp server for server-to-server transfer.

Happy hosting

JoeG

----------

## Wilhelm

What you should do imo is make a howto and post it as the first posting and elaborate your progress and/or add your scripts to automate the process.

Would be kewl if you could make enough scripts so you could pour it into a web interface.

I'm low on time and i've got exams and my efforts on this matter are for a corporate production server so they want it all custom set up to there specs and secrecy to stop known attacks.

Other than that i'm learning scripting as i go and i'm breaking open the big bad world of regular expressions and shit.

Just popped by to say that a solid thread on virtual hosting would be a fantastic thing for Gentoo since we could provide a full hosting solution and attract popularity. I f your efforts are good enough they might put you in the docs section.

Things you could address which would be very valued since i couldn't find any info on them is the following.

- Quota support

- Domains in mysql preferably linked to the users so you can dump a complete domain and users if you wish. Using a cro-job to hourly update all dns mappings. Even better would be adding complete hosting settings and service templates from mysql but that would require more research and shitloads of testing.

- Web interface (if you can build this into the main site that would rock)

Anyway you can make it as beautifiul as you like.

----------

## JoeG

The general plan is to:

1.  Get the after-howto options scripted

2.  Script the entire installation process

3.  Add options for DNS, Spamassassin, and anti-virus, chrooting postfix...other?

4.  Get help setting this all up in some graphical language, i.e. PHP (preferable) or TCL/TK...etc.

Hopefully when the scripting groundwork is set up, the GUI options will fall into place rather nicely.  It's a big project and I also have little spare time to devote to it.  But come hell or high water, I plan to finish.  Hey...on the bright side, I'm really ramping up my learning of programming.    :Very Happy:    Learning more about my system is what got me Gentoo'ing to begin with.

JoeG

----------

## Wilhelm

You should checkout PostfixAdmin. I have it running and it's sweet.

www.high5.net

----------

## JoeG

I'll check that out.  Thx for the linkage.

JoeG

[edit]Hey, I had an afterthought.  Did you ever finish those BIND scripts?  If so, couldja post?

----------

## Jaxom

ahhhh, this is something I had been looking for, admin scripts for the mail guide  :Smile: .  Personally, I prefer to keep loads seperated on each box.  IE web on a web only server, mail, on a mail only server.  But these scripts look like I could move them around where I need them.

I don't know jack about code, but I'm more than happy to assist in any way that I can.  Testing, and the like is about all I can do....but I'm currently learning php and perl....but that'll be a while yet.

----------

## JoeG

 *Jaxom wrote:*   

> Personally, I prefer to keep loads seperated on each box.  IE web on a web only server, mail, on a mail only server.  But these scripts look like I could move them around where I need them.

 

That's precisely why I separated the one for virutal mail users and virtual web sites.  Be great if you could give feedback on using these on separate boxes.  Many eyes and all...

JoeG

----------

## Chris W

 *JoeG wrote:*   

>  *Quote:*   Isn't it possible to use one squirrelmail for all users as any other service provider would do?!? 
> 
> Absolutely.  They'd just log in on the main site (not the hosted one) and use username@virtualdomain as the username.  My solution just allows for a more personalized feel for each hosted domain. 

   Have a look at the vlogin plugin for Squirrelmail.  It derives the mail domain from the web site URL... users get webmail through their own domain.  A bit fiddly to set up, but well worth it.

----------

## Wilhelm

Well i now have a fully functional virtual mailingsystem running and am busy documenting it so i can reinstall it later within a few hours.

Unlike you guys i'm setting up hosting and i'm making it highly scalable and automated. Since you can run a fully functional virtual domain and Virtual mailsystem on one system with not all that high specs i'm limiting the ammount of domains per box. When i need more space i zonk another box ontop and install the whole 8 miles and add one simple forward to my main DNS box and voila i attached another 100 domains or so to my network. I'm calling it (Gentoo Hosting Units) GHU's  :Smile: 

This way you work modularized and if one box crashes or gets hacked all other boxes continue to work there own domains. With your services-servers approach you will have a big problem if that service goes down since it will impact all users. Also if you grow beyond your wildest dreams you will have to somehow upgrade your system on the fly without downtime and scrap your old but still fully functional server.

Another big plus is i can supply different hosting schemes like fully backuped, unbackuped, mail-only because you can limit the services per box or get a better box like a dual-CPU with RAID and UPS to host expensive sites on. You wouldn't want people buying non-backupped hosting and geting backupped for free.

But then again with my approach i'm wasting HD space by installing multiple GHU's but HD space is cheap  :Smile: . Also all changes need to be done on all boxes when i do do a change. However i can add a development/testing GHU to test my new setup/modifications on.

Only minor problem i see coming is the fact that all my phpmyadmins, squirrelmails, postfixadmins need to be talked to on the correct box. This can be done easily by symlinking them to the clients domain. So the users of test2.kom can do  http://www.test2.kom/phpmyadmin/ to get their phpmyadmin. DNS will take care of the rest. Every box will have a default domain which i can use to administer them  :Smile: .

----------

