# Awstats: Phase 2 runs at snails pace

## weedy

I believe it started after the upgrade from 6.7 -> 6.9, but now hell freeze over before Phase 2 finishes.  

Any ideas? It used to plow through everything at 9000+ lines/second 

```
time /var/www/fake/cgi-bin/awstats.pl -showsteps -showdropped -config=fake.net –update -LogFile=bah.log -updatefor=1048576 

Create/Update database for config "/etc/awstats/awstats.fake.net.conf" by AWStats version 6.9 (build 1.919) 

From data in log file "bah.log"... 

Phase 1 : First bypass old records, searching new record... 

Direct access to last remembered record has fallen on another record. 

So searching new records from beginning of log file... 

8192 lines processed (1000 ms, 8192 lines/second) 

16384 lines processed (2000 ms, 8192 lines/second) 

24576 lines processed (3000 ms, 8192 lines/second) 

32768 lines processed (5000 ms, 6553 lines/second) 

40960 lines processed (6000 ms, 6826 lines/second) 

49152 lines processed (7000 ms, 7021 lines/second) 

57344 lines processed (8000 ms, 7168 lines/second) 

65536 lines processed (9000 ms, 7281 lines/second) 

65539 lines processed (9000 ms, 7282 lines/second) 

Phase 2 : Now process new records (Flush history on disk after 8000 hosts)... 

73731 lines processed (387000 ms, 21 lines/second) 

81923 lines processed (994000 ms, 16 lines/second) 

90115 lines processed (1377000 ms, 17 lines/second) 

98307 lines processed (1759000 ms, 18 lines/second) 

... 

... 

1032192 lines processed (82586000 ms, 12 lines/second) 

1040384 lines processed (82920000 ms, 12 lines/second) 

1048576 lines processed (83243000 ms, 12 lines/second) 

Jumped lines in file: 65536 

Found 65536 already parsed records. 

Parsed lines in file: 1048576 

Found 8 dropped records, 

Found 10 corrupted records, 

Found 0 old records, 

Found 1048558 new qualified records. 

 

real 1387m31.086s 

user 4m15.603s 

sys 1m18.628s
```

adjusting the logging it looks like it pauses every few lines. is this a awstats thing or could it be kernel related? 

```
4226 lines processed (140000 ms, 30 lines/second) 

... 

4240 lines processed (145000 ms, 29 lines/second) 

... 

4480 lines processed (150000 ms, 29 lines/second) 

... 

4608 lines processed (151000 ms, 30 lines/second) 

... 

4736 lines processed (161000 ms, 29 lines/second) 

... 

4992 lines processed (166000 ms, 30 lines/second) 

... 

5120 lines processed (171000 ms, 29 lines/second) 

... 

5376 lines processed (182000 ms, 29 lines/second) 

... 

5632 lines processed (187000 ms, 30 lines/second) 

... 

6016 lines processed (197000 ms, 30 lines/second) 

... 

6144 lines processed (202000 ms, 30 lines/second) 

6146 lines processed (202000 ms, 30 lines/second) 

... 

6152 lines processed (203000 ms, 30 lines/second) 

... 

6400 lines processed (213000 ms, 30 lines/second) 

... 

6552 lines processed (224000 ms, 29 lines/second) 

... 

7040 lines processed (240000 ms, 29 lines/second) 

... 

7168 lines processed (250000 ms, 28 lines/second) 

... 

7194 lines processed (255000 ms, 28 lines/second) 

... 

7424 lines processed (265000 ms, 28 lines/second) 

... 

7680 lines processed (270000 ms, 28 lines/second) 

7682 lines processed (270000 ms, 28 lines/second)
```

----------

