log4net RollingLogFileAppender: is this possible to set limit for TOTAL size of all log files?

[KellyLynch]

In my .NET application I use log4net with the following configuration:

<configuration>
<configSections>
<section name="log4net"
type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"
/>
</configSections>
<log4net>
<appender name="RollingLogFileAppender"
type="log4net.Appender.RollingFileAppender">
<lockingModel type="log4net.Appender.FileAppender+MinimalLock"/>
<file value="logs\" />
<datePattern value="yyyy-MM-dd.lo\g"/>
<staticLogFileName value="false" />
<appendToFile value="true" />
<countDirection value="1"/>
<rollingStyle value="Composite" />
<maxSizeRollBackups value="10" />
<maximumFileSize value="5MB" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger
[%property{NDC}] - %message%newline" />
</layout>
</appender>

<root>
<level value="ALL" />
<appender-ref ref="RollingLogFileAppender" />
</root>
</log4net>
</configuration>



It produces a set of log files like the following:
2012-09-24.log.0
2012-09-24.log.1
...
2012-09-24.log.10

This is ALMOST what I want. The final thing I want is: to set a limit by
total size of ALL the log files in the given location (folder). In other
words - if total size of all the log files in the given location exceeds,
say, 100MB, the oldest file will be removed; does not matter does the file
belong to current day or to some day in the past. To the moment the .config
above provides a different logic: limit by size of all the log files in the
current GROUP (in my case the 'group' is 'current day').

Can I have RollingLogFileAppender working the way I want?

[Dominik Psenner]

Hi Kelly,

Configured like this the total maximum size of all log files should be
around 5MB * 10, which is somewhat like 50MB. Scaling either the size of a
single logfile (maximumFileSize) or the number of logfiles
(maxSizeRollBackups) by 2 should bring you to your 100MB, doesn't it?

Cheers

-----Original Message-----
Sent: Monday, September 24, 2012 11:16 AM
Subject: log4net RollingLogFileAppender: is this possible to set limit for
TOTAL size of all log files?
<configuration>
<configSections>
<section name="log4net"
type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"
/>
</configSections>
<log4net>
<appender name="RollingLogFileAppender"
type="log4net.Appender.RollingFileAppender">
<lockingModel type="log4net.Appender.FileAppender+MinimalLock"/>
<file value="logs\" />
<datePattern value="yyyy-MM-dd.lo\g"/>
<staticLogFileName value="false" />
<appendToFile value="true" />
<countDirection value="1"/>
<rollingStyle value="Composite" />
<maxSizeRollBackups value="10" />
<maximumFileSize value="5MB" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger
[%property{NDC}] - %message%newline" />
</layout>
</appender>
<root>
<level value="ALL" />
<appender-ref ref="RollingLogFileAppender" />
</root>
</log4net>
</configuration>
2012-09-24.log.0
2012-09-24.log.1
...
2012-09-24.log.10
This is ALMOST what I want. The final thing I want is: to set a limit by
total size of ALL the log files in the given location (folder). In other
words - if total size of all the log files in the given location exceeds,
say, 100MB, the oldest file will be removed; does not matter does the file
belong to current day or to some day in the past. To the moment the .config
above provides a different logic: limit by size of all the log files in the
current GROUP (in my case the 'group' is 'current day').
Can I have RollingLogFileAppender working the way I want?
--
View this message in context: http://old.nabble.com/log4net-
RollingLogFileAppender%3A-is-this-possible-to-set-limit-for-TOTAL-size-of-
all-log-files--tp34471537p34471537.html
Sent from the Log4net - Users mailing list archive at Nabble.com.

[KellyLynch]

Hi Dominik ,

'should bring you to your 100MB, doesn't it?' - Unfortunately, no . If I set
maximumFileSize=10MB and maxSizeRollBackups=10, I will get 10 files for a
<Day> with total size 100MB. During the next day (<Day+1>) I will get
another 10 files for the <Day+1> with total size 100MB; the previous 10
files (ones for the <Day>) will NOT be removed when new files (for the
<Day+1>) start appearing.

I really tested the Appender in such situation.

So, the limit affects EACH SINGLE DAY; but not the whole bunch of .log files
in teh given folder

Post by Dominik Psenner
Hi Kelly,
Configured like this the total maximum size of all log files should be
around 5MB * 10, which is somewhat like 50MB. Scaling either the size of a
single logfile (maximumFileSize) or the number of logfiles
(maxSizeRollBackups) by 2 should bring you to your 100MB, doesn't it?
Cheers

-----Original Message-----
Sent: Monday, September 24, 2012 11:16 AM
Subject: log4net RollingLogFileAppender: is this possible to set limit for
TOTAL size of all log files?
<configuration>
<configSections>
<section name="log4net"
type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"
/>
</configSections>
<log4net>
<appender name="RollingLogFileAppender"
type="log4net.Appender.RollingFileAppender">
<lockingModel type="log4net.Appender.FileAppender+MinimalLock"/>
<file value="logs\" />
<datePattern value="yyyy-MM-dd.lo\g"/>
<staticLogFileName value="false" />
<appendToFile value="true" />
<countDirection value="1"/>
<rollingStyle value="Composite" />
<maxSizeRollBackups value="10" />
<maximumFileSize value="5MB" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger
[%property{NDC}] - %message%newline" />
</layout>
</appender>
<root>
<level value="ALL" />
<appender-ref ref="RollingLogFileAppender" />
</root>
</log4net>
</configuration>
2012-09-24.log.0
2012-09-24.log.1
...
2012-09-24.log.10
This is ALMOST what I want. The final thing I want is: to set a limit by
total size of ALL the log files in the given location (folder). In other
words - if total size of all the log files in the given location exceeds,
say, 100MB, the oldest file will be removed; does not matter does the file
belong to current day or to some day in the past. To the moment the .config
above provides a different logic: limit by size of all the log files in the
current GROUP (in my case the 'group' is 'current day').
Can I have RollingLogFileAppender working the way I want?
--
View this message in context: http://old.nabble.com/log4net-
RollingLogFileAppender%3A-is-this-possible-to-set-limit-for-TOTAL-size-of-
all-log-files--tp34471537p34471537.html
Sent from the Log4net - Users mailing list archive at Nabble.com.

[Dominik Psenner]

Post by KellyLynch
If I set
maximumFileSize=10MB and maxSizeRollBackups=10, I will get 10 files for a
<Day> with total size 100MB. During the next day (<Day+1>) I will get
another 10 files for the <Day+1> with total size 100MB; the previous 10
files (ones for the <Day>) will NOT be removed when new files (for the
<Day+1>) start appearing.
I really tested the Appender in such situation.
So, the limit affects EACH SINGLE DAY; but not the whole bunch of .log files
in teh given folder

Now I got your point. However, keeping files of past days once the rolling
runs by date is a feature documented also at [2]. There it says:

-- quote --
This example show how to configure the RollingFileAppender to roll log files
on a date period and within a date period on file size. For each day only
the last 10 files of 1MB will be kept.
-- eof quote --

Since you only want to limit the file count anyway, it makes sense to not
roll by date but by file size, doesn't it? Doing so gives you the wanted
behaviour, but filenames won't contain the date. Sorry for the
inconvenience.

Clearly the above is only a workaround and a date in the filename would be
nice, but it is not trivial to implement such a rolling without being
computationally expensive.

The problem lies in identifying the files that should be deleted even if
those files are many thousands or their filename contains a date that is
years in the past. One can either probe a filename or probe if there is a
file for a specific date. Both strategies do not fit into the current
implementation of the rolling file appender. There are several known
problems with the rolling file appender and a rewrite is in discussion. To
document this discussion I've CC'ed the dev mailing list.

If you believe that you need this as a feature, please leave a feature
request in JIRA at [1]. Of course it is no guarantee that one of the
developers will work on this issue any time soon, and therefore I encourage
you to actively work on it if you need it soon. Patches that improve log4net
are always welcome!

Best regards,
Dominik

[1] https://issues.apache.org/jira/browse/LOG4NET
[2] http://logging.apache.org/log4net/release/config-examples.html