What's new in version 1.2.1?

Configuration file environment variable substitution: Strings of the format {VAR} can now be used in configuation files. When processing the file, Reptor will replace the string with the contents of the environment variable VAR. This allows fairly complex changes to be made to the configuration from the command line, which can be useful if you're scripting Reptor in a loop to process multiple files or multiple reports.
RBL Summary: A new report has been added that summarizes RBL activity. The new configuration file option rbl_summary controls this feature.
Bandwidth alert: Reptor can be configured to generate an alert message for connections that exceed a given average bandwidth over the life of the connection. This can be specified with the new configuration file option bandwidth.
Bandwidth display in alerts: The average bandwidth for the life of the connection can be displayed in the alerts report with the new show_bandwidth option.
Inclusion of configuration in HTML output as a comment: The HTML output now includes the contents of the configuration file that was used to generate it.
Progress meter.: A crude progress meter, which can be useful when running Reptor manually, has been added. It is based on the time of day, so it will be accurate only when processing complete single-day logs. This feature is controlled by the --progress command line option. Don't use this when invoking Reptor from cron!
Better recognition of Mobile users: Reptor will now recognize Mobile users that have @ in their userid.
Better handling of HTTPS URLS: Reptor will now tag HTTPS URLs as "unknown" in order to avoid empty values in reports.

What's new in version 1.2.0?

Arbitrary pattern matching

Reptor can now generate a simple report for any text in the logfile that matches arbitrary patterns. This can be used to count occurrences of certain strings that Reptor doesn't otherwise report directly on, such as "Possible Port Scan detected on Interface" or similar warnings. This feature is controlled by the pattern option. Use of this feature can become somewhat complicated -- please see the sample reptor.cfg file for a more detailed explanation and examples.

VPN summary

A new report has been added that summarizes VPN usage. The new configuration file options vpn_summary and vpn control this feature.

Configuration file includes

Configuration files can now be included from other configuration files. This can be handy when you have more than one firewall that you're running Reptor against, and you don't want to maintain multiple configuration files that are mostly the same. The settings that are shared between each firewall can be placed into one common configuration file, and then included from the smaller firewall-specific configuration files. The include option controls this feature.

Bugfixes

The host column in the host_www_summary, protocol_host_detail and user_host_detail reports are now DNS resolved if the dns option is specified.

No output will be generated for a report section if that section has a zero total.

The raw interface name will now be used to identify an interface if no alias has been configured for it. This prevents blank entries in reports indexed by interface, and removes the need to even define the interfaces at all.

The history_summary graphs now start at "1 day ago" instead of "0 days ago", and fill in from right to left as the data file builds over time.

Email output to more than one recipient has been fixed.

What's new in version 1.1.0?

Subnet filtering

Reports can now be generated only for activity that occurs on specific subnets. The new configuration file option netmask controls this feature. This feature allows you to create separate reports for different subnets. To do so, you will have to create multiple configuration files, and run Reptor on each separately (with the same logfile).

Host WWW summary

A new report has been added that shows WWW usage by host. For each host, the most popular web sites visited from that host will be summarized. It is very similar to the user_www_summary report, and is useful in situations where users do not authenticate with the firewall. The new configuration file option host_www_summary controls this feature.

MIME type alert

Reptor can be configured to generate an alert message for connections that are denied based on MIME type. This can be specified with the new configuration file option mime.

DNS resolution changes

The dns all feature has been removed. You shouldn't have been using it anyway. If you were, use jdresolve-raptor instead. The previous functionality of the dns print option is now specified simply by dns.

SEF v7 "missing proto tag" bug

Added a work-around for a bug in SEF v7 that sometimes causes lines to be logged without a proto tag.

Bugfixes

Removed the need to have an end-of-line character on the last non-comment line of the configuration file.

User strings that contain spaces are now parsed properly.

What's new in version 1.0?

OOBA Summary

A new report section has been added which will display information regarding out of band authentication. See the configuration file option ooba_summary.

Workaround for Raptor SMTP received bytes bug

Reptor can work around a bug in Raptor that occasionally produces impossibly huge values for the number of bytes received in some logfile entries for the SMTP protocol. This workaround is disabled by default and can be enabled with the new configuration file option smtp_bugfix.

DNS caching

Reptor now maintains a local cache of DNS names, in order to avoid redundant (and probably slow) lookups. This will dramatically speed up processing if you're using "dns all", which you shouldn't be doing. If you're using "dns print", the impact is not as significant.

New HTML themes

Four new HTML themes have been added: "Aqua" (based on the MacOS X GUI), "Hotmail2" (based on the new Hotmail design), "Slashdot" (based on slashdot.org), and my new favorite, "Slashcode" (based on slashcode.com). Themes may not render correctly if your browser is deprecated and does not properly support modern HTML standards. All themes produce valid HTML 4.01 Transitional output and have been verified to work properly with Internet Explorer 5.0, 5.5, and 6.0, Mozilla 0.9.4, and Opera 5.12. I've had enough of trying to get the themes to render properly with Netscape 4.x. If you insist on using it, don't email me to complain about how crappy your output looks.

Bugfixes

Fixed the "dns print" bug that caused processing to run for extremely long times when that option was specified and DNS resolution was not enabled on the firewall. (It was actually functioning the same way that "dns all" does.)

Fixed a bug that caused the alert section user column to show the string "$user" instead of the actual user name.

What's new in version 0.99?

Automatic handling of split logfiles

Woohoo! Finally, Reptor can automatically handle large logfiles that have been split by the firewall and named logfile.date, logfile.date-1, logfile.date-2, etc. This new feature can work with files retrieved by remotelog and even in conjunction with the uncompress configuration file option, in the case that each logfile chunk is individually compressed.

basedir option

The --basedir command line option or the basedir configuration file option may be specified to indicate the Reptor install directory. This may be helpful when running Reptor from a command scheduler, so that it can properly locate the required files. If both the command line and configuration file options are specified, the command line option takes precedence. If --basedir is specified and your configuration file is named reptor.cfg, you won't need to specify the --config command line option.

HTML themes

Reptor now uses a simple theme system to format the output. This makes it very easy to change the look of the reports if so desired, without the need for editing the main program itself. A number of sample themes are included with the distribution. Selecting a theme is done with the new configuration file option theme. As a result of this new feature, the following configuration file options have been removed: table, highlight, and style_sheet. All included themes produce "HTML 4.01 Transitional" compliant output, which has been tested to render properly with Internet Explorer 5.5, Netscape Navigator 4.76, Opera 5.02, and Mozilla 0.8.

Note: If you make your own theme and send me a copy, I will include it in the base distribution of the next release. Please make sure your theme produces "HTML 4.01 Transitional" compliant output.

User WWW summary

A new report has been added that shows WWW usage by user. For each user, the most popular web sites they have visited will be summarized. This option is only useful if users authenticate with the firewall. The new configuration file option user_www_summary controls this report.

--history command line option

If you've specified the history_file option in your configuration file, Reptor will normally only update history when processing the previous day's logfile, to avoid contaminating the history file in the case that you process a logfile more than once. Specifying the --history command line option will override this behavior, and Reptor will update the history regardless of the logfile being processed. It is up to you to keep the history file in date order.

Support for Raptor Mobile v6.5

Reptor will now detect activity from Raptor Mobile version 6.5 clients in the mobile_summary section.

WebNOT fetcher summary

A new section has been added that shows the status of the WebNOT fetcher process. It will report if the ratings file is up to date or if a new one has been been installed. This report section is controlled by the new configuration file option fetcher_summary.

Minor text formatting updates

Reptor can now print large numbers comma formatted. This optional formatting is controlled by the configuration file option nice_format, which replaces the nice_bytes option.

Reptor will now print "Gb" instead of using huge "Mb" values.

Minor bug fixes

The uncompress configuration file option now works in conjunction with the --log command line option.

Wide tables are now formatted correctly by Netscape. (This wasn't a bug in Reptor, but work-around for the issue has been discovered.)

What's new in version 0.98?

Protocol detail by host

A new report section has been added that shows traffic for a single protocol, grouped by host. Multiple sections can be added, each reporting on a different protocol. Such a section could answer the question, "What hosts generate (or serve) the most ftp traffic?" or "What hosts generate (or serve) the most smtp traffic?"

Protocol detail by user

A new report section has been added that shows traffic for a single protocol, grouped by user. Multiple sections can be added, each reporting on a different protocol. Such a section could answer the question, "What users generate (or serve) the most ftp traffic?" or "What users generate (or serve) the most smtp traffic?" Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

User detail by host

A new report section has been added that shows traffic for a single user, grouped by host. Multiple sections can be added, each reporting on a different user. Such a section could answer the question, "What hosts is Joe visiting (or serving to)?" or "What hosts is Sam visiting (or serving to)?" Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

User detail by protocol

A new report section has been added that shows traffic for a single user, grouped by protocol. Multiple sections can be added, each reporting on a different user. Such a section could answer the question, "What protocols is Joe using?" or "What protocols is Sam using?". Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

Host detail by protocol

A new report section has been added that shows traffic for a single host, grouped by protocol. Multiple sections can be added, each reporting on a different host. Such a section could answer the question, "What protocols is 192.168.1.1 using?" or "What protocols is 192.168.1.2 using?"

Host detail by user

A new report section has been added that shows traffic for a single host, grouped by user. Multiple sections can be added, each reporting on a different host. Such a section could answer the question, "What users are accessing 192.168.1.1?" or "What users are accessing 192.168.1.2?" Note: You must be performing user authentication at the firewall in order for the user data to appear in the log file.

Bug fixes

Fixed a Unix bug where a single digit day is reported without zero padding.

Fixed a possible divide by zero error in the history summary.

Fixed a bug where the message summary would default to no messages instead of all messages when a severity limit was not specified.

Minor usability enhancements

Removed the Netscape-choking font size definitions from reptor.css.

Added better checking for invalid configuration file options.

Fixed some HTML formatting inconsistencies.

What's new in version 0.97?

Message summary

Reptor can now print a summary of messages other than statistical information (type 121). This can be useful for determining, at a glance, possible abuse of the firewall. For example, possible port scans (type 347), unauthorized protocol commands (type 334), or attempts to access control ports (type 515) might warrant further investigation.

Historical activity summary

Reptor can now accumulate daily activity statistics in a history file. A summary has been added that graphs the contents of this file. Alternatively, it could be easily graphed with external programs such as gnuplot or Excel.

Color highlighting

Reptor can now highlight alternating table lines with different colors. This is done with style sheet classes (called odd and even) for the TD attribute.

Minor usability enhancements

Reptor no longer requires an explicit "+" on ascending sort identifiers in the configuration file.

Reptor will exit with an error if it encounters an unprocessable configuration file directive.

Bug fixes

Fixed a spelling error.

Fixed the mail output subject line when processing cut logfiles.

Fixed a bug where the user summary wouldn't work unless the alert section was also included.

What's new in version 0.96?

Logfile compression

Reptor can now read compressed logfiles. The decompression program can be specified with the uncompress configuration file option. Also, files that are saved (as specified by the save_logfile option) can be compressed. The program to perform this compression can be specified with the compress option.

Support for cut/merged logfiles

The firewall creates each new logfile with a specifically formatted first line. Previously, Reptor required this line to be present. This caused problems if logfiles were cut into multiple pieces or if multiple logfiles were merged into a single large one. Reptor now specifically supports these situations by not requiring this special first line, and gracefully adapting the output if it is not present. A new command line option --ignore can be specified to indicate that the logfile to be processed has been cut or merged. This will cause Reptor to not look for the first line, and ignore it if present. This feature allows Reptor to report on a time period less than one day by feeding it a cut logfile, or on a time period greater than one day by feeding it merged logfiles.

Normally, if output file or output ftp is specified without a filename, Reptor will use the datestamp from the first line of the logfile to automatically generate one. If this situation exists when the --ignore option is specified, Reptor will default to a filename of "reptor.html".

Verify save_logfile directory

If the save_logfile option is specified, Reptor will verify that the specified directory actually exists.

What's new in version 0.95?

GNU GPL: Reptor is now released under the GNU General Public License.
User summary: A new summary has been added that shows usage by user. This is typically only useful if users authenticate against the firewall.
Top level domain summary: A new summary has been added that shows usage by top level domain.
Raptor Mobile summary: A new summary has been added that shows the number of connections made by Raptor Mobile clients. Since there is no way to determine when the tunnel was closed, it is not feasible to report on the tunnel traffic itself -- only the number of connections made is shown.
Save local copy of logfile: If you're using remotelog to retrieve logfiles, you can specify the new save_logfile option in the configuration file to indicate that you want Reptor to make a local copy of the logfile for additional analysis or backup.
New path parameter for remotelog option: If you are using the remotelog utility to retrieve logfiles from your firewall, and the remotelogfile executable is not in your shell's PATH, you'll need to specify this so Reptor knows where to find it.
Check for remotelogfile program: If the remotelog option is specified in the configuration file, and Reptor is run with the --verify command line parameter, Reptor will abort with an error if the remotelogfile program does not exist or is not executable.
Passive FTP transfers: The FTP output now uses PASV mode for compatibility with Raptor v6.
New options for email output: New mail_from and mail_server options in the configuration file allow you to specify the from address and SMTP relay if you're using email output. This may be necessary to avoid problems with anti-spam features of some email servers.
Style sheet option: A new style_sheet option in the configuration file allows you to specify a style sheet to be embedded in the report.
Bug fix: Fixed code to recognize directory names ending with \ and not just /.
Version option: A new --version command line option has been added. Guess what it does.

What's new in version 0.91?

Support for Raptor Firewall version 6: Reptor now understands the format of logfiles generated by Raptor Firewall version 6.
WebNOT ratings alert: Reptor can be configured to generate an alert message for connections that trigger a WebNOT ratings denial. This can be specified with the new configuration file option ratings.
New fields available in alert section: The alert section can now optionally contain the logfile fields "rule", "id", "op", and "result". These are specified with new configuration file options show_rule, show_id, show_op, and show_result.

What's new in version 0.90?

Extended configuration file options: Almost all command line options have been replaced with configuration file options. No more huge command lines! A few command line options remain in order to allow overrides of the configuration file settings. These remaining options have been changed to the GNU style long format. For example, -d is now --date. Refer to the About Reptor page for details.
Automatic local/remote host discovery: The configuration file local option has been removed in favor of the interface option. Instead of providing a list of IP addresses that are considered to be "local", you must specify what network interfaces exist in the firewall server. Reptor will then automatically determine which hosts are local and which are remote based on what interface they reside behind.
Traffic filter by interface and direction: Traffic can now be filtered by direction and by interface. If you have more than two network interfaces in the firewall server, you may want Reptor to ignore certain traffic, such as "from the DMZ interface to the outside interface" or "from the inside interface to the DMZ interface".
Remotelog support: Reptor can now obtain logfiles through the remotelog utility that is provided with the firewall. This allows Reptor to be run on a machine other than the firewall server without the need for customized scripting to transfer the logfiles, and without the fear of the logfile content being sniffed from the network.
Total summary: A new simple summary has been added that shows total traffic through the firewall.
Alert summary: A new simple summary has been added that shows how many alerts each host has triggered. This makes misuse tracking a bit easier -- a host that triggers two or three alerts is probably nothing to worry about, but a host that triggers fifty seven calls for further scrutiny.
Time of day summary: A new summary has been added that shows the amount of traffic based on the time of day.
Time of day alert trigger: Logfile entries can now trigger alerts based on the time of day, and can be separately defined for each protocol.
Graphs: Summary tables can now optionally include bar graphs to indicate traffic as a percent of total.
Configurable word search fields: You can now specify which fields are included in the word search. The string that the word search scans can be built from any combination of source host name, remote host name, and connection argument. (The connection argument is the value from the logfile "arg" field, which typically contains the URL.)
Reverse DNS lookups minimized: Reverse DNS lookups can now optionally take place only if a host name actually gets included in the output. Doing this is dramatically faster than performing a lookup on every unresolved IP address, and only slightly slower than not doing any reverse lookups at all. However, the drawbacks of this option are that the hostname will not be able to be included in the word search string, and that summaries will not be able to be sorted by hostname.
Default style sheet: The HTML output now includes settings defined by a default style sheet, to allow easy customization of report aesthetics.
Optional hyperlinks: Hyperlinks are now optional. New links have also been added before each report section to allow easy navigation.
Automatic version detection: Reptor will now automatically determine the firewall version you are using.
Time alert renamed: The time alert option has been renamed to duration.
Size alert renamed: The size alert option has been renamed to volume.
Bug fixes: Summaries now correctly count the limit of number of entries, and durations greater than 23:59:59 will now display correctly.
Additional optional alert section fields: The alert section may now contain additional optional fields, individually selectable. These include: the reason for the alert, the user, the argument of the connection, and the authentication method used.
Removed non-HTML output formats: The plain text and delimited text output format options have been removed.
Allow aliases with spaces: Aliases may now contain spaces.