# Dont use metalog!!!

## Hackeron

I was attracted to "metalog is a logger with high performance" comment on the gentoo install guide. I installed it on my new server and it ran just fine for several weeks - although there is definitely a lack of features compared to syslog-ng.

Anyway, suddenly out of the blue, no modifications or anything, it started eating up all my resources! -- I checked "top" which took over 10 minutes to start up! and it wouldnt show any irregularity so I did the windowy thing and restarted machine. What do you know, it hung on startup!!!

Since the machine has no monitor or keyboard, I had to bring it upstairs and try to figure out what the fuck happened! -- its been running for several weeks without being touched at all, and suddenly it fails. -- naturally I assumed hardware and starting changing over components.

It took me about 5 hours to figure out metalog was to blame, and my machine would freeze on startup until I did rc-update del metalog and rebooted without rescue liveCD. Also, as soon as I would start metalog, my machine would start to lag terribly, but top still doesnt show metalog to take more than 2% cpu! -- so very strange.

Anyway, point is *DO NOT USE METALOG*!

syslog-ng offers all the features metalog does, has a much clearer syntax, supports logging of un matched things rather than having to tell every single filter the same rules and takes virtually no resources.

Here are my configurations for syslog-ng:

```
options { long_hostnames(off);  sync(0);  stats(43200);};

source src { unix-stream("/dev/log"); internal(); };

source kernsrc { file("/proc/kmsg"); };

destination messages { file("/var/log/messages"); };

destination kernmsg { file("/var/log/kernel"); };

#destination console_all { file("/dev/tty12"); };

log { source(src); destination(messages); };

log { source(kernsrc); destination(kernmsg); };
```

----------

## forceflow2

dunno, I've never had any problems with it

----------

## Mit

Never had any issues with metalog, its running (and has been) on my server since i installed it, which would be now over 2 years (hardware isn't the same, but nm about that).

Sometimes it might be an idea to some looking around before yelling that something isn't any use like that. I'm not saying there never anyhting wrong with Metalog, but i'll bet more users have had no problems than have had problems with it.

----------

## ben_h

It's your fault, not metalog's. I've run it for generations.

----------

## LordArthas

Hi!

I'm running metalog as well, as per Gentoo AMD64 Handbook, and I have no problems.

Michele.

----------

## Hackeron

I also had no problems, then it suddenly fails. Cant figure out how or why. I havent changed anything at all for 2 weeks.

In anycase, why use metalog if it can just fuck over for no reason unexpectedly? -- Please explain how that is my fault.

 *Quote:*   

>  I'm not saying there never anyhting wrong with Metalog

  There you go, metalog is just bad. Why would any logger ever have problems at all -- weeks after working properly!

Also, why use metalog if it only lacks features to syslog-ng -- check out their forum on sourceforge and see for yourself. You want simplicity get syslog, you want features, get syslog-ng -- why get metalog? -- it makes no sense, stop making people fall victim to your poor recomendations. 

Here is my metalog configuration that worked for weeks:

```
maxsize  = 5242880

maxtime  = 604800

maxfiles = 5

Firewall :

  facility = "kern"

  minimum  = 6

  regex = "Shorewall"

  logdir = "/var/log/metalog/firewall"

Kernel messages :

  facility = "kern"

  logdir   = "/var/log/metalog/kernel"

  neg_regex = "Shorewall"

Crond :

  facility = "cron"

  logdir   = "/var/log/metalog/crond"

Password failures :

  regex    = "(password|login|authentication)\s+(fail|invalid)"

  regex    = "(failed|invalid)\s+(password|login|authentication)"

  regex    = "ILLEGAL ROOT LOGIN"

  logdir   = "/var/log/metalog/pwdfail"

FTP Server :

  program  = "proftpd"

  logdir   = "/var/log/metalog/ftpd"

SSH Server :

  program  = "sshd"

  logdir   = "/var/log/metalog/sshd"

Everything important :

  facility = "*"

  minimum  = 6

  logdir   = "/var/log/metalog/everything"

  neg_regex = "Shorewall"

Everything very important :

  facility = "*"

  minimum  = 3

  logdir   = "/var/log/metalog/critical"

console logging :

  facility = "*"

  command = "/usr/sbin/consolelog.sh"
```

----------

## Hackeron

From metalog's sourceforge page:

```
- Bugs ( 6 open / 6 total )

Bug Tracking System

 - Support Requests ( 0 open / 0 total )

Tech Support Tracking System

 - Patches ( 1 open / 1 total )

Patch Tracking System

 - Feature Requests ( 10 open / 10 total )

Feature Request Tracking System
```

100% of reported bugs open, 100% of feature requests ignored, and even a submitted patch that is ignored. 

*DO NOT USE METALOG*!

----------

## forceflow2

Sounds like one of the syslog-ng developers is getting jealous...

----------

## Hackeron

 *forceflow2 wrote:*   

> Sounds like one of the syslog-ng developers is getting jealous...

  lol, I'm not a syslog-ng dev  :Smile:  -- I dont even know C (yet).

----------

## forceflow2

Ok...you could be a hired hand, sent forth to spread the evil of metalog.   :Wink: 

----------

## Mark Clegg

I had the same - metalog fine for ages, then all of a sudden, all the CPU is taken. I'm on sysleg-ng now - and it's fine.

----------

## Hackeron

 *Mark Clegg wrote:*   

> I had the same - metalog fine for ages, then all of a sudden, all the CPU is taken. I'm on sysleg-ng now - and it's fine.

  Thank you! -- glad I'm not alone here  :Smile: 

Anyway, warning to anyone planning or currently using metalog. Cant stress it enough: Do not use metalog!

----------

## Redeeman

OMG!! DONT USE A COMPUTER!!!!!!!!

it works fine... for a long time..

but ALL OF A SUDDEN!!!! it goes insane!!!!!!

it breaks!!!!1 ohh dear god no!

the motherboard vendors ignores me when i say i dislike the grey color! NOOOOOOOOOOOOOO!!!!!!!!

and omg no ffs, when i suggest they redesign their print layout, i get ignored too!!!!!! and the worst is, they dont even wanna add my feature suggestion, which is support for both intel and amd cpu

FFFFFSSSSSSS STOP USE IT!!!!!!!!!!!!!!!!!!!!!!

<similar example with a car>

....

----------

## Irrumator

Yeah it does the same for me d00d!! what will we do??!?!onetwothree?

----------

## Redeeman

 *Irrumator wrote:*   

> Yeah it does the same for me d00d!! what will we do??!?!onetwothree?

 

dear god! im glad im not alone on this!

simply. DONT USE A COMPUTER!!!! move the bits in your head instead, a computer only lacks features!

in your head, you can imagine anything!!!oneone!!!1111!

and besides, when spoken of features!!! man invented computer!!!!!!11111111!!!!!1

----------

## Hackeron

 *Redeeman wrote:*   

>  *Irrumator wrote:*   Yeah it does the same for me d00d!! what will we do??!?!onetwothree? 
> 
> dear god! im glad im not alone on this!
> 
> simply. DONT USE A COMPUTER!!!! move the bits in your head instead, a computer only lacks features!
> ...

  lol

----------

## ryker

I have used syslog-ng in the past without problems.  On my last firewall/router setup, I decided to try metalog because it was suggested in the Gentoo install instructions.  I haven't had any problems with it, but know I am worried.  I'm considering removing metalog and installing syslog-ng after reading this thread.  Can anyone tell me why I might choose to stick with metalog instead of using syslog-ng?

----------

## Valhlalla

Becuase of the pathetic FUD campain against it.

[edit] I'm not a real big fan of metalog, but it's been working with me for over a year. No reason to change.

----------

## llsardonicll

Almost sounds like mudslinging political propaganda... Metalog is for cancer!

Anyway, I use metalog and I've had no problems

----------

## Hackeron

 *ryker wrote:*   

> I have used syslog-ng in the past without problems.  On my last firewall/router setup, I decided to try metalog because it was suggested in the Gentoo install instructions.  I haven't had any problems with it, but know I am worried.  I'm considering removing metalog and installing syslog-ng after reading this thread.  Can anyone tell me why I might choose to stick with metalog instead of using syslog-ng?

  If you would've asked me a week ago, I would be saying "your fault, works for me" or pointing out the excess level of emotion like Redeeman (nice one btw) or just dismissing it as FUD. The fact is many are not having problems with it, but you have 2 examples of it being able to take down a potential critical server.

Yes, I did present the problem like a petty child, but what would you say when its the reason for 2 days downtime of a critical server? - I had the feeling of finding the idiot that recommended metalog in the first place and beating the crap out of him -- instead, I used capitals and exclamation marks   :Smile: 

The biggest problem with my experience is it failed after 2 weeks of working just fine, and I havent touched the PC during that time. I posted my configs and I used the latest stable version with nothing masked or hardmasked on the system at all. There are no objections to my config and even if someone analysis and find something potentially wrong (any takers?), why then did it work for 2 weeks?

Again, I cant stress enough there is no political or personal preference. I switched to syslog-ng strictly because metalog took down a critical server for a reason I'm yet to figure out, and I soon found out syslog-ng is superior in every aspect - Where are the politics? -- I dont work for syslog-ng, I dont know C, and I like the metalog community as they helped me with some problems on the forums. 

After experiencing critical problems with it myself however, I found out while metalog is supposed to be a production level, stable piece of software, it has several experimental changes made to it on the CVS every so often with no QA and 100% of the reported bugs (some intermittent like mine) are just ignored. -- Certainly not something you want to have on a production server, and concidering that syslog-ng no longer lacks features in comparison to metalog, Im just giving a well substantiated warming to users and potential users of metalog.

----------

## waverider202

metalog is a high performing syslog server.  Why is it high performing?   Its high performing cause it buffers the log entries for a period of time.  It does occasional flushes to your hard disk.  This delayed hard disk activity makes metalog perform a lot better than any other system logger.  This is why people use it.

As for a critical server.....you wouldn't want to use metalog.  If something goes wrong and your box goes down, you wouldn't know what happened.  With the other loggers, you'd know exactly what went wrong.

For metalog itself, did anyone try running a trace on the program to see why it was taking so much resources?  I've had occurences where X would randomly spin off into oblivion.   I've had other software do the same thing from time to time.  It sounds like this was a recuring event for the people that did have this issue, so what ways did you try to debug metalog itself?

Metalog doesn't get developed a lot, but how much does it need it?  Yes, there are spiratic occurences where the service spins out of control, but if a developer can't replicate the problem, then they can't do anything about it.  As for feature requests....its fast because it has no features.  Its one feature that it has over every other logger increases performace at the expense of log integrity during a system crash.

----------

## Hackeron

 *waverider202 wrote:*   

> metalog is a high performing syslog server.  Why is it high performing?   Its high performing cause it buffers the log entries for a period of time.  It does occasional flushes to your hard disk.  This delayed hard disk activity makes metalog perform a lot better than any other system logger.  This is why people use it.

  The default configuration these days disables the buffer and you can use a buffer with syslog-ng as well and it has a lot more control like logging to a database, etc. Infact you can use any logger to log to tmpfs or memfs and occationally flush so you can see whats going on in real time and still let drive spin down.

And really, think about it.. If the machine is busy, metalog will access drive every minute or so anyway. If the machine is not busy, you may get to see your critical messages only several days after the logger gets them (or never at all).

 *Quote:*   

> As for a critical server.....you wouldn't want to use metalog.  If something goes wrong and your box goes down, you wouldn't know what happened.  With the other loggers, you'd know exactly what went wrong.

   First of all the default config now stops this, but really, if you want to use this behaviour you lose both the ability to debug system crash and lose the ability to see logs in real time. Seems like about as much of a tradeoff as simply not having a logger at all.

 *Quote:*   

> For metalog itself, did anyone try running a trace on the program to see why it was taking so much resources?  I've had occurences where X would randomly spin off into oblivion.   I've had other software do the same thing from time to time.  It sounds like this was a recuring event for the people that did have this issue, so what ways did you try to debug metalog itself?

  There is no X on that machine, and no I didnt try strace as the base system is compiled with -fomit-frame-pointer -- Who would've expected a critical bug in something as basic as a logger?

What I did do is disable the logger all together with liveCD (after having to put a cdrom drive in the machine), atleast then the machine booted. Then I simply started metalog with /etc/init.d/metalog start and my system started to lag badly until I stopped it. Nothing in logs, nor can anyone point something wrong in my configs.

 *Quote:*   

> Metalog doesn't get developed a lot, but how much does it need it?  Yes, there are spiratic occurences where the service spins out of control, but if a developer can't replicate the problem, then they can't do anything about it.  As for feature requests....its fast because it has no features.  Its one feature that it has over every other logger increases performace at the expense of log integrity during a system crash.

  Ah, its fast because it has less features? --  I agree to that, but that goes against the description in the install guide or portage: "A highly configurable replacement for syslogd/klogd"

Maybe that ought to be changed to "a stripped down performance logger with buffer".

Also, there are bug reports, some you can reproduce yourself, others give detail were things go wrong and what code is likely to blame. I suppose I could run strace and add yet another bug report.

----------

## forceflow2

OMFG BUGX0RS!!!

MY SOFTWARE ISN'T COMPLETELY INFALLIBLE?!?!?!?!?!? NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!

^^What your posts read as to me.

----------

## Hackeron

 *forceflow2 wrote:*   

> OMFG BUGX0RS!!!
> 
> MY SOFTWARE ISN'T COMPLETELY INFALLIBLE?!?!?!?!?!? NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!
> 
> ^^What your posts read as to me.

  Monkey see, monkey do. Redeeman already said that and then Irrumator repeated, now you.

I most certainly expect the system logger not to kill my system. But where did I say I expect it to be completely infallible?

Redeeman uses the most experimental kernel patches and uses all hard masked software -- so ofcourse I expect him to break my balls about it, its all in good fun.

I however use strictly software marked stable on this server and I do infact expect rock stability from the software I use, or I wouldnt be using it. -- Various bugs are fine, but we are talking about a bug in the /logger/ that /kills/ the system , not a bug in a graphical toolkit that makes a box not always draw properly or a bug in application X that crashes if you do X -> Y -> Z..

You are using the Microsoft logic that if there are more bugs in linux software that in windows software it makes windows software better. Its not the amount, its the severity. 

I'm talking about something you set up and expect to work for a decade despite any bugs not worked out -- a bug fix is more likely to introduce more bugs than it fixes. Redeeman with his hourly kernel re-compiles isnt familiar with the concept -- and bless him, let him find bugs in bleeding edge software - who knows he may find a security bug that applies to my version as well.

Now, if you are one of those infamous ricers, you are accustomed to instability in software marked stable, and think its normal. You may not understand this, and not much I can do but point you to http://funroll-loops.org/ -- but I do expect stability like most linux users.

----------

## forceflow2

I like monkeys

Oh...also, I run the unstable arch (~x86) and haven't had many show stopper problems like what you describe...anyways...this topic was moot from the beginning.

----------

## ryker

This topic was definetely not moot.  This is exactly what the forums are for.  Everyone contributes with his/her experiences, opinions, and recommendations.  I think even Hackeron would agree that metalog probably works fine for most people.  He is just contributing his experience with metalog, and giving his opinion based on his experience.  I haven't seen a properly configured syslog-ng kill any of my machines, nor have I seen any forum posts to this affect.  I may not even remove metalog from my machine.  At least now if I have a similiar performance problem, I know to at least consider that it might be the logger.  The logger isn't something you normally have to worry about.

----------

## Hackeron

 *ryker wrote:*   

> This topic was definetely not moot.  This is exactly what the forums are for.  Everyone contributes with his/her experiences, opinions, and recommendations.  I think even Hackeron would agree that metalog probably works fine for most people.  He is just contributing his experience with metalog, and giving his opinion based on his experience.  I haven't seen a properly configured syslog-ng kill any of my machines, nor have I seen any forum posts to this affect.  I may not even remove metalog from my machine.  At least now if I have a similiar performance problem, I know to at least consider that it might be the logger.  The logger isn't something you normally have to worry about.

  Thanks for that  :Smile: 

I do agree, metalog /does/ work for most people, hell it worked for me. As I said, if someone would've posted this experience a week ago, I would also be saying "works for me".

Why I even bother with this thread is because the "top" utility didnt even show metalog to take more than 2% cpu at any time. While as soon as I would start it, system would start to lag terribly and return to normal as soon as metalog was stopped.

Thats what made it so hard for me to even figure out metalog is to blame, and I had to install a cd drive on the machine, reboot with liveCD, try disabling all services ironically except the logger and just sitting there completely puzzled why on earth the system freezes on boot. -- I even tried removing udev in favour of devfs.

Removing metalog solved the problem and it seems hardly reasonable for a logger to take down the system on any configuration, much less several weeks after - so just a friendly warning.

----------

## fleed

I agree with Hackeron on this thread. I had been using syslog-ng previously on a different distro but switched to metalog because the gentoo install docs point to it. I used it for a few months, maybe close to a year but it's lack of features and weirdness started taking it's toll and I just switched to syslog-ng. Now I'm happy again. I think if you want a simple logger just use the plain syslog. If you want a good and light one use syslog-ng. It's only drawback is the complexity of the configs but that's no excuse for not using it!

----------

## zeek

I have Metalog running on 20+ servers, which are providing DNS, HTTP and SMTP for a very active network.  The email load alone is ~5 million msg/day.

Metalog has never given any problems, in fact it is one of the most robust pieces of software I have ever had the pleasure of running.

So if Metalog is indeed causing this "slowdown", where are the relevant diagnostics like strace output, gdb back traces, etc?  This entire thread is pure FUD.

The original posters problem is, to put it bluntly, somewhere between the keyboard and chair.

----------

## zeek

It looks like you already have a problem, and it is documented in this other thread:

https://forums.gentoo.org/viewtopic.php?p=1675329&highlight=#1675329

Except in that thread its "Udevd eating resources", not metalog.

----------

## Dr_Stein

Just don't log anything. Dump it all to /dev/null or to a fifo. 

Who needs logs anyway.  :Wink: 

(yes, I'm kidding.. so laugh!)

----------

## nobspangle

There is one very good feature of metalog, automatic log rotation. Works well for me.

I installed syslog-ng on one box because the gentoo handbook changed it's recommendation, but the way it stores the log files is archaic and I quickly switched to metalog. I also don't understand what's compilcated about the configuration.

```
title

what you want to log

where you want to log it to
```

easy

----------

## Spiralis

 *zeek wrote:*   

> I have Metalog running on 20+ servers, which are providing DNS, HTTP and SMTP for a very active network.  The email load alone is ~5 million msg/day.
> 
> Metalog has never given any problems, in fact it is one of the most robust pieces of software I have ever had the pleasure of running.
> 
> So if Metalog is indeed causing this "slowdown", where are the relevant diagnostics like strace output, gdb back traces, etc?  This entire thread is pure FUD.
> ...

 

I have had metalog running on 4 servers:

2x 1,5 years

1x 1 year

1x 0.5 year

I have had no problem whatsoever, but it is interesting to read about people that has had problems. The problem with the original poster is that there is no documentation other than "i turned off metalog an w00ps everything is fine". I guess that it is a somewhat more complex problem here. 

So, I agree with you zeek, the problem exists between the chair and the keyoard...  Or the familiar 40 cm problem (40 cm from the screen) 

...untill futher documentation is given  :Smile: 

----------

## revertex

for more than a year i used metalog without a problem, re-reading the gentoo install handbook i notice that it suggest syslog-ng, then i give a try to syslog-ng...my bad, time to come back to metalog, it simple work out of the box, no complex config needed, perfect for lazy ppl like me.

----------

## Doomhammer

I've use metalog for months now, on two separate Gentoo boxes with completely different hardware, and everything is fine...  :Confused: 

----------

## g4c9z

Original post said:

 *Quote:*   

> syslog-ng offers all the features metalog does, has a much clearer syntax

 

Are you serious!?  Metalog's syntax is straightfoward.  As a programmer, I can honestly say that I've found syslog-ng's syntax more confusing that a programming langauge!  I wouldn't mind using it if I could just figure out how!

Plus, metalog is the only logger that handles file permissions properly!  The other 2 loggers expect you to write the permissions of each log file in a configuration file and they use those permissions, ignoring the ones that are already on the files!  Metalog simply gives files full permissions and lets you restrict the permissions of the folder they're going in by using chmod normally!

Now, if it does indeed have bugs, that's another issue.  But I've been using it for awhile now and it seems to be working fine.  I understand that there may well be a major bug in it, and that possibility is almost enough to make me switch to syslog-ng despite its flaws.  But it's also quite possible that the experience was due to something else.  Frankly, I don't trust any piece of software, let alone syslog-ng, unless I wrote it myself - so switching from metalog wouldn't really help matters that way.

It is annoying that metalog seems abandoned, though, or so it seems from its website.  I'm guessing its author started using syslog-ng too...  :Smile: 

----------

## blueribbon

 *g4c9z wrote:*   

> 
> 
> It is annoying that metalog seems abandoned, though, or so it seems from its website.  I'm guessing its author started using syslog-ng too... 

 

Well... it works, does what's supposed to do.  :Smile: 

----------

## g4c9z

 *Quote:*   

> Well... it works, does what's supposed to do. Smile

 

Prove it.

And having it run for 10 years without problems is not a proof.  :Smile: 

That's why I'm not convinced syslog-ng doesn't screw up once in a long while too, like metalog seemed to do.

----------

## foosh

 *Hackeron wrote:*   

> Anyway, warning to anyone planning or currently using metalog. Cant stress it enough: Do not use metalog!

 

Metalog works brilliantly for me and has since I started running Gentoo on this computer about 1.5 years ago.  

But just to humor you:  

Don't use metalog?  What would you recommend as a replacement and why exactly would your suggested replacement outperform metalog?

----------

## blueribbon

 *g4c9z wrote:*   

>  *Quote:*   Well... it works, does what's supposed to do. Smile 
> 
> Prove it.
> 
> And having it run for 10 years without problems is not a proof. 
> ...

 

I'm not saying it doesn't have a bug. I'm just saying that it's plain simple and plain effective. I like simple things, I don't want to learn a strange configuration format just to configure syslog-ng. And can live with a small bug (that I don't know about, yet)

----------

