# Overview of how I should best set up a file server?

## OS Newbie

Ok, all you Linux masters,

I don't mean to have anyone launch into the details.  But I want to set up a file server for a small LAN, where I can have users of Mac, Linux and Windows machines write files that may be needed by other users or by they themselves on any of the other platforms.  For instance, say that a bunch of sound files get digitized by Mac, Linux and Windows-based machines, and some Mac user has done a lot of graphics work, creating images using things like Photoshop, Illustrator and Painter.  Say that somebody wants to take any or all of those files, and make something with them (like a film), or maybe wants to use Final Cut, to make a narrated slide show.  Another thing that would likely happen is that a programmer would want to create a database application to be the "big jukebox in the sky" that would allow people to play any of the sound files from a menu.  In any case, you get the idea.  It would be great to be able to set up directories for various kinds of files, and have them be accessible to Mac, Windows and Linux platform users.

I understand many of the the issues of file format compatibility between platforms.  So I don't need to hear anything about that.  What I'm wondering here is whether or not Linux in general (and Gentoo in particular) is the way to go, to create such a server, and if so, what pieces are needed to do it.  Should I set up a Samba-kind of thing, make everything look like Windows, or what?  I wish it were as easy as it used to be when one could bring up a Novell server, and make it look like a Mac hard drive to Mac users, while simultaneously make it look like a Windows hard drive to Windows users.  Well, actually, now that I think about it, there were more interplatform file format compatibility issues then.

Anyway, if you have any general thoughts, recommendations, etc., I'm all ears.  Thanks in advance.

----------

## John R. Graham

Split from Old computer hacker (but Linux newbie) seeks guru... as it really is (and deserves) its own topic.

- John

----------

## 1clue

OK so if this were me with the project, I have 3 possible alternatives and different reasons for them.  The order I give is the order of my preference.  Keep in mind I went through the first thread with OS Newbie too, so I kind of know what you're after.  Please read before getting angry.

One:  Buy a NAS.

Reasons:

Features.  If you look at these things, they're not just file servers with RAID and an easy control panel, but they're also media servers for your (or your kids' or grandkids') ipods and ipads and Android devices, and a host of other things that someone our age just doesn't think of, but it's there.

Easy.  Hook up some disks, turn it on, go to the web page configuration, go to town.

Price.  If you consider your time to be worth anything, then you can't build a one-off for even close to the cost of these things, feature for feature.  NAS devices are extremely competitive right now.

Time.  I have way too many projects and not nearly enough time, and this would be WAY down on the list.

Linux.  Most of these things run Linux or some other free OS anyway, with a few closed-source packages in there for good measure.

Two:  A NAS distro.

Reasons:

Somebody already thought of the things a good modern NAS needs, and they've set up a Linux distro (or FreeBSD or whatever) specifically for it.

You still build your own hardware.

You have more control over what's on the basic system.

You get easy configuration through a control panel.

Some of the NAS devices on the market are based on these distros anyway.

Three:  Full-on DIY from source.

Reasons:

Build your own hardware.

Install software piecemeal.

You control everything on the system.

You learn the most.

I'll put my approach into another post.

----------

## OS Newbie

 *1clue wrote:*   

> OK so if this were me with the project, I have 3 possible alternatives and different reasons for them.  The order I give is the order of my preference.  Keep in mind I went through the first thread with OS Newbie too, so I kind of know what you're after.  Please read before getting angry.
> 
> One:  Buy a NAS.
> 
> Two:  A NAS distro.
> ...

 

I'm not going to get angry.  Thanks for responding.  Of your three choices, I would pick number one.  And I may actually do that.  But I'd really prefer to avoid buying anything at all.  And I don't care the least bit about things like RAID, or any features at all, really.  I'd like to simply make the drive space available to these different platforms, without a care in the world about anything that I don't need to care about.

I tried freeNAS, but found it to be way too complicated to configure, or even to understand.  I'd bet that dedicated NAS boxes would seem simpler.  But rather than go to that solution, I'll probably just connect a USB-tethered drive to each machine in turn as needed, and simply use sneakernet.  That requires absolutely no muss nor fuss.  I presume that I could format the drive NTFS, and have it be readable on any of the three platforms.  Is that the case? (how's that for a Linux newbie question?)

@John R. Graham: That brings me to another point.  I realize that this deserves its own thread.  I thought about that before I appended it to the old one.  But my inclination was to limit the scope of the population being questioned, to those who had been participating in my OS newbie seeks guru conversation.  I was thinking that they had probably already become at least somewhat accustomed to my nature, and that because of that, I might avoid needlessly irritating some folks.  Your call though.

----------

## Jaglover

You install Gentoo.

Decide where your shared volume will be, say /mnt/shared.

You export /mnt/shared via NFS, will work for Linux and OS-X.

You can share it via AFP for Mac if you like it more.

You share /mnt/shared using Samba/CIFS, for Windows machines.

It is irrelevant what filesystem is used.

You can keep sharing it over different protocols, like http for read only access, ftp, whatnot.

----------

## 1clue

NTFS:  You're going to want a good NTFS driver for your Mac.  Linux works well enough.

I'm a big fan of sneakernet too, I have a slot-load SATA drive cartridge bay in most of my server hardware.  I have enough bandwidth problems on my network.

----------

## OS Newbie

 *Jaglover wrote:*   

> You install Gentoo.
> 
> Decide where your shared volume will be, say /mnt/shared.
> 
> You export /mnt/shared via NFS, will work for Linux and OS-X.
> ...

 

Thanks, much!  That is exactly what I had in mind.

----------

## OS Newbie

 *1clue wrote:*   

> NTFS:  You're going to want a good NTFS driver for your Mac.  Linux works well enough.
> 
> I'm a big fan of sneakernet too, I have a slot-load SATA drive cartridge bay in most of my server hardware.  I have enough bandwidth problems on my network.

 

I like the simplicity of it, and the fact that I don't have to "share" any drives over the network.  That eliminates some concerns about vulnerability.

As for the NTFS driver (for the Mac), yes that's true.  I'd forgotten about that.  But I already have that covered anyway, just to make it possible to attach roving NTFS drives to a Mac when I want to deal with some Windows thing (on a Mac).  Why I was thinking of using NTFS on the shared server drive was because the server space would look "common" to Windows and Mac machines, thus making things simpler from the views of Windows and Mac users.  I could still obviously get to the drive from Linux machines.

----------

## creaker

I would install samba.

1. Windows, Mac and almost all the linux distros has built in protocols to connect to samba server. For Gentoo clients you have to install a small piece of software - cifs-utils

2. No matter what a filesystem do you use at server - ext, ntfs or something else - for clients it's just hidden under the hood. They operates with shares as with regular native partitions.

3. No need for any kind of drivers (ntfs or whatever else)

4. As for me, the main advantage of samba is ability to play movies and music directly from server. No need to download them to client before starting playback. FTP doesn't allow such a thing, you have to download file previously. Assuming the same with http and nfs protocols, though I'm not pretty sure.

----------

## OS Newbie

 *creaker wrote:*   

> I would install samba.
> 
> 1. Windows, Mac and almost all the linux distros has built in protocols to connect to samba server. For Gentoo clients you have to install a small piece of software - cifs-utils
> 
> 2. No matter what a filesystem do you use at server - ext, ntfs or something else - for clients it's just hidden under the hood. They operates with shares as with regular native partitions.
> ...

 

That sounds ideal.  But I've wondered about is whether or not the various clients would be happy with files that they see under Samba.  You say that clients operate with shares the way they do with native partitions.  Does that mean that to Macs, files look like they are on say, a "Mac OS Extended (Journaled)" partition, and Windows machines think that they're talking to say, an NTFS partition?  Can these different machines share server drive space, each seeing it as its own native format?  If so, how does Samba deal with the inherent differences in things like file attributes?  Or do you need to give each platform its own "share" area?

----------

## John R. Graham

No. All files accessed on a Samba share look like they're on a Windows filesystem, which is the downside for the *nix clients: file permissions aren't fully correct for the *nix model. However, if your Windows clients all use a sufficiently up-to-date version of Windows (7 and later), an NFS client is included (just not installed by default). That means that you can have a unified solution for all clients.

- John

----------

## creaker

 *OS Newbie wrote:*   

> 
> 
> That sounds ideal.  But I've wondered about is whether or not the various clients would be happy with files that they see under Samba.  You say that clients operate with shares the way they do with native partitions.  Does that mean that to Macs, files look like they are on say, a "Mac OS Extended (Journaled)" partition, and Windows machines think that they're talking to say, an NTFS partition?  Can these different machines share server drive space, each seeing it as its own native format?  If so, how does Samba deal with the inherent differences in things like file attributes?  Or do you need to give each platform its own "share" area?

 

You can't operate with shared partition in terms of low-level filesystem structure. You can't resize, recreate, formatting, defragmentation or any other low-level actions that you can do at the local partitions. All other operations (files and directories creation/deletion/movment/copying) can be done in a usual for the current system way. Without dedicating personal share for each system. Of course filees/directories operations should respect current permissions.

No need for additional drivers and third party tools for getting access to share (e.g. ext driver for Windows & mounting/un-mounting tool)

----------

## Atom2

 *John R. Graham wrote:*   

> However, if your Windows clients all use a sufficiently up-to-date version of Windows (7 and later), an NFS client is included (just not installed by default). That means that you can have a unified solution for all clients.

 I wish you were right, but unfortunately that's not correct - at least not for Windows 7 (I have to admit that I do not have any information about Windows 8 as I do not use that system): For Windows 7 only the Ultimate and Enterprise versions of Windows 7 do actually include an option to install the NFS client. Not even the Professional version which (due to its option to enable domain joins) is by far the most common version in SMB environments, does include the NFS client.

Given that the Enterprise version is only available for larger corporations under a volume license model, Windows Ultimate is the only option for home users and SMBs should they require NFS client support.

Atom2

----------

## 1clue

Following the "what could possibly work on everything" path is unsatisfactory.  There's always something that doesn't work, and most of it doesn't work very well.

In order to make this seamless enough for the tech-unsavvy in your family or friends group is to have a share for each platform.

Macs CAN share NFS but it's not really all that friendly about it.  Mac wants their particular breed of networking, especially if you want a Time Machine backup.  Even though they CAN go to a cifs share or an nfs share, they generally don't show them when you browse the network, unless you know how.

I know that it's possible to make the same partition on the server look like NFS and CIFS and AFP, because I've seen it on commercially available NAS devices.  According to the apple forums, the drive should be HFS+ shared over AFP, and Bonjour for time machine discovery.  They also use Bonjour for media servers and networked audio systems, probably more.  I'm sure there's some way for the same partition to look like the proper thing to each platform, and that includes Android and ios devices.  

This link might give you an idea about the Apple side of it:  http://blog.west.uni-koblenz.de/2013-04-24/timemachine-backup-of-mac-on-a-linux-fileserver/

----------

## OS Newbie

 *John R. Graham wrote:*   

> No. All files accessed on a Samba share look like they're on a Windows filesystem, which is the downside for the *nix clients: file permissions aren't fully correct for the *nix model. However, if your Windows clients all use a sufficiently up-to-date version of Windows (7 and later), an NFS client is included (just not installed by default). That means that you can have a unified solution for all clients.
> 
> - John

 

That was what I expected would be the case.  Isn't there a third party NFS client for other versions of Windows 7?  I have a couple of laptops that run W7Home 64-bit.  I doubt that I'll ever go buy a copy of Windows in any form.

----------

## 1clue

Interesting.  I'd never have a Windows that wasn't pure straight from Microsoft.  I only use it for testing software that customers will use from Windows.

Yes, there are lots of NFS drivers, some of them are good, and most of those cost money.  Which one is best depends on the phase of the moon IMO and I haven't looked for awhile so you have to ask somebody else.

----------

## OS Newbie

 *1clue wrote:*   

> Interesting.  I'd never have a Windows that wasn't pure straight from Microsoft.  I only use it for testing software that customers will use from Windows.
> 
> Yes, there are lots of NFS drivers, some of them are good, and most of those cost money.  Which one is best depends on the phase of the moon IMO and I haven't looked for awhile so you have to ask somebody else.

 

If I can avoid it (and there's really no reason why I can't), I will never again own or run a single line of Microsoft code.  Well, that's not really completely true.  I'll keep my two Windows 7 laptops for awhile, and a few old XP machines (until I get Gentoo sorted out on them).  But the sentiment is there.

I expect never to have any customers who will run any of my code at all, at least knowingly, much less from Windows.  I would just want the NFS drivers for use on my own LAN, so that I can easily move between various platforms while having access to common files.

----------

## Roman_Gruber

 *OS Newbie wrote:*   

>  *1clue wrote:*   Interesting.  I'd never have a Windows that wasn't pure straight from Microsoft.  I only use it for testing software that customers will use from Windows.
> 
> Yes, there are lots of NFS drivers, some of them are good, and most of those cost money.  Which one is best depends on the phase of the moon IMO and I haven't looked for awhile so you have to ask somebody else. 
> 
> If I can avoid it (and there's really no reason why I can't), I will never again own or run a single line of Microsoft code.  Well, that's not really completely true.  I'll keep my two Windows 7 laptops for awhile, and a few old XP machines (until I get Gentoo sorted out on them).  But the sentiment is there.
> ...

 

The thing is, do you use Skype? Since they got their hands on it I dissbanded it. Skype hardly worked before but after they took over Skype it goes worst. Want a mobile phone? I won t buy NOKIA bc it is bought my that company.

It is a pain already to be forced to use their horseshit at work so I keep far away from that and when someone asks me for advise I say sorry, I do not use that thing for years. No idea how it works and I am not htat interested at all.

For private life except gaming there is no need for that OS anyway

----------

## 1clue

@OS Newbie,

If you can reasonably dictate which operating systems connect to your home network then your task is much easier and the commercial NAS approach loses its appeal.

Frankly the only mainstream consumer operating system that doesn't currently connect to my network on a semi-regular basis is iOS.  That's random chance AFAIC.  The people who connect want all the bells and whistles.

@tw04l124,

Skype is a huge problem for me.  It doesn't work on Linux at all and it works marginally on Macs.  I use it extensively for my work, and my wife uses it privately to talk to her family.  I'm desperate for a replacement.  We need text messaging personally and in groups, voice personally and in groups, and desktop sharing personally and in groups.  Sometimes during a meeting we point a camera at a whiteboard for video.  My wife uses video a lot.

----------

## Spanik

Interesting thread, have been thinking about such a thing for some time. But something hasn't been touched and that is power consumption. Now what would be the most economical? I understand that in a commercial setting this would be second to reliability. But for home use power consumption is important (well, to me at least).

How would a Gentoo setup fare against a commercial NAS in that aspect? I'd prefer a Gentoo setup to a bought NAS for the learning experience. What would be the minimal requirements that result in low power with acceptable performance for such a file server? Let's say for 4 people with enough spread use. So not streaming 4 HD while at the same time doing other stuff, Gb ethernet available.

----------

## 1clue

@Spanik,

Normally I'd say this should be a different thread, but somehow I think OS Newbie would want to go there.

The commercial NAS vendors have a lot going for them.  They've already done a LOT of research (more than you could justify for a single box, or even a dozen of them) into hardware and found something they like.  They've done a LOT of research into software and have come up with something that works best with that hardware.  It's unlikely IMO that your hardware selections and your software configuration would come up with something that beats theirs in all ways, especially if you include cost into the mix.

Most of the SOHO NAS products are running Linux or FreeBSD under the covers, so technically speaking if you come up with hardware you like and that hardware is fully compatible with a Linux kernel, then you should be able to do as well as the NAS vendors aimed at the SOHO market would if they were using the same hardware you chose.  As well, your priorities may be different from what the NAS vendors used, and because of that your home-brew NAS might match your specific needs much better than the commercially available options.

One thing I would emphatically say is stay away from WD green drives on Linux.  I followed all the instructions and had 6 at one point, only one survivor left, the other 5 died within 9 months of purchase.  I'm giving WD a break altogether for a few years, though I doubt they'll notice.   :Smile: 

If you factor "learning experience" into it as any sort of significant factor, then I say go build your NAS from scratch.  If your focus is just to get a cost effective NAS with all the standard bells and whistles -- especially if you have a flock of other people who want iOS and Mac and Windows and Android and whatever-kind-of-TV-that-is, and blu-ray and media servers -- then I say go buy a NAS.  By the time you've finished, if you're a full time employee with a decent income, you can't possibly do all that as cheaply as the NAS vendors do.

----------

## OS Newbie

 *1clue wrote:*   

> @Spanik,
> 
> Normally I'd say this should be a different thread, but somehow I think OS Newbie would want to go there...
> 
> 

 

Yeah, but then, I'd probably be happy staying in my original thread for years, the one about seeking a guru.  Maybe I should create a forum for the express purpose of carrying on this conversation about being a curmudgeonly old gray-beard who bristles at the thought of being a newbie at anything, but finds himself in that position just because Linux is his only way out of the trap that is the ever-more-dumbed-down proprietary OS.

As for the NAS VS homebrewed solution, I've decided that I will divide the problem into parts.  My main interest in having a file server (as a programmer, anyway) is to make all the technical issues having to do with incompatibilities between various platforms disappear.  I don't want anybody to have to think about any things like file attribute compatibilities/incompatibilities between OS X, Windows, Linux, Unix, etc., as I construct shared applications (like a big jukebox/media server and its associated repository of media files and the database that goes with it).  And since the W3C has shot Microsoft between the eyes (not that they've realized it completely), That feat can be easily accomplished by engineering the whole enchilada as a web application.  All the clients, be they Mac, Windows, OpenBSD or what, can interact with my software the same way, without any platform-specific issues.  That solves my main problem about file system compatibilities.

Other reasons for me to have a file server include things like providing shared directories for users, backup space in separate buildings here, etc.  And for things like those, I can live with the limitations of having platform-specific areas, using NFS, Samba, and AFP.  I don't care about things like Time Machine backup at all.  So that won't matter.  If people can't be bothered to place backup copies of things that are important to them, onto servers for protection, then I can't be bothered to even hear about any losses that they suffer.

So, I will forgo any benefits of having NAS units, and also quit thinking about whether or not transparent file sharing can be made to work between OS X, Windows and Linux users.

----------

## 1clue

I'm speculating here, but I suspect that the NAS units probably use HFS+ volumes underneath.  It seems to me that HFS+ has more sophisticated permissions than the others, which might mean that NFS and CIFS could fit inside HFS+, so to speak.

Again, speculation.

Really I think you don't want it to be time machine anyway, time machine takes over the entire drive (or partition in this case) and keeps stuffing data in until you run out, then deletes the oldest to make enough room for the next one.  It's really a backup-only solution.

----------

## causes

I'm also interested in setting up my server computer to work as network storage. Currently i'm using just sshfs. I wish that the network storage would work just like locally mounted filesystems. Seems like i have to look into samba or NFS?

----------

## OS Newbie

 *1clue wrote:*   

> I'm speculating here, but I suspect that the NAS units probably use HFS+ volumes underneath.  It seems to me that HFS+ has more sophisticated permissions than the others, which might mean that NFS and CIFS could fit inside HFS+, so to speak.
> 
> Again, speculation.
> 
> Really I think you don't want it to be time machine anyway, time machine takes over the entire drive (or partition in this case) and keeps stuffing data in until you run out, then deletes the oldest to make enough room for the next one.  It's really a backup-only solution.

 

I'll be constructing all of the shared apps that will be used by anyone and everyone on my network.  And since I'll do so using web-based technologies, I can use whatever disk partitioning/volume formats that I wish.  And given that, it makes sense to use NFS or some other Linux/Unix scheme, since the apps will be running on a Linux or Unix server.  The users, be they running Unix, Linux, Mac, Windows or what, will interact with the server only through my apps.  So it will be only my apps that will deal with disk I/O.

And I agree that Time Machine isn't worth considering (in my world).  That, along with most other Apple apps can go die the death of 10,000 screams, as far as I'm concerned (and so can their authors, and the marketing honchos behind them).  If any users want to run Time Machine, they're free to do so using drives that are physically connected to their workstations.  I don't want to know about it.

----------

## OS Newbie

 *causes wrote:*   

> I'm also interested in setting up my server computer to work as network storage. Currently i'm using just sshfs. I wish that the network storage would work just like locally mounted filesystems. Seems like i have to look into samba or NFS?

 

Reference Jaglover's post of September 29th in this thread.  What he describes is in a nutshell, what you want to do.

What I was sidling up to as an idea was to figure out how to make a common (to all platforms) storage area appear to each platform as though it were a native disk.  But after considering it, I abandoned that idea, and decided that giving the users of each platform their own respective areas would suffice.  That's just because I don't want to even bother to explore the issues pertaining to file attribute differences, etc.  I can write (web-based) applications that are accessible to all the platforms, and eliminate the requirement from all of them to read and write from/to the disk themselves.

----------

## OS Newbie

 *Spanik wrote:*   

> Interesting thread, have been thinking about such a thing for some time. But something hasn't been touched and that is power consumption. Now what would be the most economical? I understand that in a commercial setting this would be second to reliability. But for home use power consumption is important (well, to me at least).
> 
> How would a Gentoo setup fare against a commercial NAS in that aspect? I'd prefer a Gentoo setup to a bought NAS for the learning experience. What would be the minimal requirements that result in low power with acceptable performance for such a file server? Let's say for 4 people with enough spread use. So not streaming 4 HD while at the same time doing other stuff, Gb ethernet available.

 

It seems to me that one could implement a low-power system, based on say, a Beaglebone Black board or something, and then attach whatever disk drives would be required for storage.  My guess would be that the disk drives would be the big power hogs.  In fact, probably a laptop could serve as the computer.  They're optimized for low power consumption.  And you could put the largest 2.5-inch internal drive available, in it.  That would probably suffice for most file server requirements.  But what do I know?  Maybe someone here has done some futzing around with this low-power server idea.

I'd like to play with this too, eventually.  So if you get anywhere with it, come back and tell us all about it.  I'm in the process of setting up a secondary electrical distribution system here.  It'll be completely off-grid (not tied).  I'll initially utilize micro-hydro generators, and later add solar panels.  So the idea of using laptops as servers (and workstations, for that matter) appeals to me because I'd be able to feed them directly from my low-voltage power system, without the need to step up to 110V and then back down.

----------

## 1clue

I don't know about beaglebone black, but I have 3 raspberry pi's.  They would make horrible file servers.  They have a single 100 mbps (not gigabit!) network card and they can't saturate it even with nothing else going on.

IMO when I drag something to a file server or vice versa, I want it done ASAP.  If it takes too long you get your workstation falling asleep and losing the transfer, or any number of other things to interrupt and disrupt.  If it's too slow nobody will use it.

These small boards don't have any hardware for simultaneous data transfer in any quantity, so I'm saying look before you buy.

That said, raspberry pi's make a really nice, really cheap stratum 1 time server when you add a GPS chip.  Or if you want some network services up all the time, like a full-fledged DNS server for a small office, they're great for that.  If I needed a Windows Domain controller for authentication on a small network I might try to set that up on a pi using samba.

From what I understand a beaglebone black is just about twice the size of a pi.  I think for a file server you at least would want an atom or equivalent, stay away from arm processors IMO.  I'd look for some sort of proof that you can run the disk at full speed and the network at full speed too.  I'd also make sure you have a decent mechanism for high speed backups.

----------

## OS Newbie

 *1clue wrote:*   

> I don't know about beaglebone black, but I have 3 raspberry pi's.  They would make horrible file servers.  They have a single 100 mbps (not gigabit!) network card and they can't saturate it even with nothing else going on.
> 
> IMO when I drag something to a file server or vice versa, I want it done ASAP.  If it takes too long you get your workstation falling asleep and losing the transfer, or any number of other things to interrupt and disrupt.  If it's too slow nobody will use it.
> 
> These small boards don't have any hardware for simultaneous data transfer in any quantity, so I'm saying look before you buy.
> ...

 

Okay, okay... I'm sorry I mentioned it.  Let's discard the idea about using any such card.  As soon as I thought about using a laptop instead, my thoughts moved completely over to that idea.  I should have stated it in that way (or even just deleted the Beaglebone black part).  That's not to say however, that I think it would be a problem to use one as a file server.  I haven't done any benchmarking.  But the last time I checked, there weren't any disk drives that had throat speeds capable of saturating the bandwidth of Fast Ethernet.  Does anyone here know what the sustained data transfer rate is for his favorite drives?  I just looked at lots of pages on the WD site, but could find a single spec on it.  All they talk about is the 6GB transfer rate of the interface, the maximum (burst) data rate of the drive, etc.

In any case, this guy was interested in low power.  And one quite probably has to make some trade-offs in performance to get power usage down significantly.

----------

