monitoring

squidanalyzer - Squid proxy log analyzer and report generator

Website: http://squidanalyzer.darold.net/
License: GPLv3
Description:
Squid proxy native log analyzer and reports generator with full
statistics about times, hits, bytes, users, networks, top URLs and
top domains. Statistic reports are oriented toward user and
bandwidth control; this is not a pure cache statistics generator.

SquidAnalyzer uses flat files to store data and doesn't need any SQL,
SQL Lite or Berkeley databases.

This log analyzer is incremental and should be run in a daily cron,
or more often with heavy proxy usage.

Packages

squidanalyzer-6.6-1.noarch [175 KiB] Changelog by Django (2017-05-07):
Add TopStorage configuration directive to limit the storage of url to
a certain quantity in data file and sorted by OrderUrl. On huge access
log it will improve a lot the performances but you will have less
precision in the top url. Default to 0, all url will be stored.
Add a cache to network and user aliases for speed improvement. Thanks to
Louis-Berthier Soulliere for the report.
 - Add TimeStart and TimeStop configuration directives to allow to
   specify a start and stop time. Log line out of this time range
   will not be parsed. The format of the value is HH:MM. These
   directives can be overridden with the -s | --start and -S | --stop
   command line options. Thanks to Louis-Berthier Soulliere for the
   feature request.
 - Add UpdateAlias configuration directive to apply immediately the changes
   made in aliases files to avoid duplicates. You still have to use
   --rebuild to recreate previous reports with new aliases. Enabling
   this will imply a lost of performances with huge log files.
 - Add UseUrlPort configuration directive to be able to include port number
   into Url statistics. Default is to remove the port information from the
   Url. Thanks to Tobias Wigand for the feature request.
 - Add report of top denied url on user statistic page. Thanks to delumerlino
   and Pavel Podkorytov for the feature request.
 - Add last visited timestamp on urls reports and show last ten visit on user
   url report. The last visit are counted after 5 minutes in hour view, after
   30 minutes in day views and per day in month view. Thanks to Ringa Mari
   Sundberg for the feature request.
 - Add support to ipv6 address dns resolving, you need perl > 5.014. Thanks
   to Brian J. Murrell for the report.
Full list of other bug fixes:
 - Change user top url title from "Top n/N Url" into "Top n/N sites". Thanks
   to Daniel Bareiro for the report.
 - Update documentation to clarify the use of space character in aliases
   files. Thanks to Darren Spruell for the report.
 - Fix explanation of UserAlias file format about ip address vs DNS name.
   Thanks to Darren Spruell for the report.
 - Fix missing report of TCP_DENIED_REPLY messages. Thanks to Jeff Gebhardt
   for the report.
 - Add license file about resources file and a script to retrieve original
   javascript libraries.
 - Fix html report building that was limited to the last day.
 - Fix missing network alias replacement.
 - Update year in copyrights.
 - Disabled bandwidth cost report by default.
 - Fix removing of obsolete year directory.
 - Fix obsolete statistics no longer being deleted. Thanks to andreybrasil
   for the report.
 - Allow parsing of access.log generated through syslog. Thanks to Celine
   Labrude for the report.
 - Add Url_Hit label in translation files.
 - Fix remaining _SPC_ in username. Thanks to roshanroche for the report.
 - Fix remaining SA_CALENDAR_SA in html output. Thanks to roshanroche for
   the report.
 - Add more fix to denied stat datafile corruption. Thanks to PiK2K for the
   report.
 - Fix denied stat datafile corruption. Thanks to PiK2K for the report.
 - Use CORE::localtime to format denied first and last hit.
 - Fix potential unparsed log case when log file are set in configuration
   file and not on command line.
 - Change the in-line popup (on top domain and top URL) to show hits on hits
   tables, bytes on the bytes tables and duration on the duration tables,
   instead of count. Thanks to Wesley Bresson for the feature request.
 - Only apply OrderUrl to user url list, other reports in Top domain and Top
   Url are now always ordered following the first column, which is the sorted
   column of the report (hits, bytes and duration).
 - Fix missing limit total number of URLs shown for a user to TopNumber.
   Thanks to Graham Wing for the report.
 - Update statistic on users with DENIED code to have the full list of
   user/ip even if they never hit an url.
 - Change Perl install directory from vendor to site to avoid well know issue
   on BSD. Thanks to dspruell for the report.
 - Add initial Debian package build files
 - Update squidanalyzer.css changed the width of the single menu tabs,
   because in German language, it looks better at the tab "TOP DENIED" is in
   German language "TOP VERBOTEN" and will be displayed better, no wordwrap
   anymore, will be done with this change. Thanks to Klaus Tachtler for the
   patch.
 - Fix Throughput label for unit/s that was not dynamically changed during
   value formating and always labelled as B/s. Thanks to aabaker for the
   report.
 - Fix typo in graph titles. Thanks to aabaker for the patch.
 - Update missing fields to German language file. Thanks to Klaus Tachtler
   for the patch.
 - Fix top url report that was not cumulate statistics anymore. Thanks to
   Wesley Bresson for the report.
 - Fix typo about Network exclusion. Thanks to Mathieu Parent for the patch.
 - Manpages fixes. Thanks to Mathieu Parent for the patch.
 - Use FHS for manpages path. Thanks to Mathieu Parent for the patch.
 - Update russian language file. Thanks to Yuri Voinov for the patch.
 - Fix typo in mime type redefinition.
 - Mark mime-types with invalid characters as "invalid/type". Thanks to
   gitdevmod for the report.
 - Add missing throughput translation entries in lang files. Thanks to Yuri
   Voinov for the report.
 - Fix major issue in squidguard and ubfguard history file managment. Thanks
   to Guttilla Elmi for the report and the help.
 - Fix path to xzcat program durinf install. Thanks to Johan Glenac for
   the report.
 - Fix auto detection of SquidGuard log file when there is no denied entry
   in the first lines.
 - Fix typo in debug messages
 - Add warning when DNSLookupTimeout is reach. Thanks to gitdevmod for the
   report.
squidanalyzer-6.5-1.noarch [168 KiB] Changelog by Django (2016-02-18):
This is a mantenance release to fix an overlaping bug on bytes charts with 
last versions of browsers like firefox, iceweasel and chrome. 
  - Fix height of bytes graphs that was overlaping on third graph. 
    Thanks to Daniel Bareiro for the report. 
  - Update russian translation. Thanks to Yuri Voinov for the patch. 
  - Update copyright year.
squidanalyzer-6.4-1.noarch [164 KiB] Changelog by Django (2015-12-21):
Add -t | --timezone and TimeZone directive to change the timezone. When set,
SquidAnalyzer will read time from log file as UTC time and will add the
hours specified in the timezone option. This is useful if the log file is
not parsed on a computer with the same timezone than the squid server.

It also included several bug fixes since last release.

  - Fix graphic overlaps that with one of the graphics. Thanks to Daniel Bareiro
    for the report.
  - Add throughput calculation (ratio between bytes and duration) to all reports.
  - Fix missing largest URL in networks detailed report. Thanks to Juan Martin
    for the report.
  - Fix use of network-aliases together with a network include entry that make
    networks disappears from the report. Thanks to Juan Martin for the report.
  - Add -t | --timezone and TimeZone directive to change the timezone. When set,
    SquidAnalyzer will read time from log file as UTC time and will add the
    hours specified in the timezone option. Thanks to Anderson - BR Suporte for
    the feature request.
  - Add support to ufdbGuard log file. squidGuard and ufdbGuard files can be
    given together with squid log file as a list into LogFile configuration
    file or as arguments of command line. Thanks to Martin Hoffmann for the
    feature request.
  - Fix some division by zero. Thanks to cueda for the report.
  - Fix some potential illegal division by zero.
  - Fix negative duration with http like log file when duration is not set (-).
    Thanks to cedua for the report.
  - Add new throughput (Bytes/sec) column in all reports and a throughput graph.
    Thanks to Mike Lerley for the feature request.
  - Allow parsing of xz compressed files. Thanks to Markus Maikis for the patch.
  - Fix bug with include/exclude networks or clients preventing users reports to
    be built. Thanks to Juan Martin for the report.
  - Fix SquidAnalyzer fails to update statistics after cleanup of access.log.
    Thanks to mkhallaf for the report.
  - Limit parsing of ufdbGuard logs to BLOCK line.
  - Replace SquidGuard label by Blocklist as we use more blocklist tools.
  - Update Italian translation file. Thanks to Stefano Cailotto for the update.
squidanalyzer-6.3-1.noarch [162 KiB] Changelog by Django (2015-10-17):
This release adds a new report to show statistics about Denied URLs. It also
allow to add a SquidGuard log to the list of log files and to report blocked
URLs into the Denied reports. It ialso adds a pie chart on SquidGuard ACLs use.

There's also four new configuration directives:

  - UserReport to be able to remove any user related reports but statistics
    about URL and domains will remain.
  - ExcludedCodes to be able to exclude some log entries following the TCP
    code returned.
  - UrlHitsOnly to be able to enable the generation of additional HTML tables
    with top Url per byte and per duration in Top Urls and Domains report.
  - MaxFormatError to not exit immediatly when a bad format error is encountered. 
    SquidAnalyzer will wait MaxFormatError before exiting.

Note that this last directive is disable by default, so if you still want the
three tables in the reports, you must set UrlHitsOnly to 1.

A Catalan translation file has been added to the lang directory.

It also included several bug fixes since last release.

  - Skip immediately lines that squid is not able to tag: TAG_NONE. Thanks to
    David Touzeau for the report.
  - Fix display order when OrderUrl was set in Top Url and Top Domain views.
    Thanks to Wesley Bresson for the report.
  - Convert fr_FR.txt translation file from ISO_8859-1 to UTF8 and change
    charset value. Thanks to zezinho42 for the report.
  - Change order in de_DE.txt of WeekDay to So Mo Di Mi Do Fr Sa, the week
    days in translation file must start with Sunday unlike in calendar.
  - Fix case sensitivity in command line options. Thanks to Pavel Podkorytov
    for the report.
  - Add SquidGuard.current state file to be able to do incremental parsing of
    both squid and squidguard log files without issues.
  - Try to fix bad characters in mime_type field and add MaxFormatError to not
    exit immediatly when a bad format error is encountered. SquidAnalyzer will
    wait MaxFormatError before exiting.
  - Add information about how to parse SquidGuard log together with Squid Cache
    access log file.
  - Add pie chart on SquidGuard ACLs use.
  - Remove redundant regular expressions.
  - Try to fix case when method or code in log file are corrupted with non
    printable characters, should never appears but some injection have been
    reported.
  - Add support for SquidGuard log parsing to report denied ACLs. Thanks to
    Pavel Podkorytov for the feature request.
  - Fix detection of new log file from history when log file was in common
    http format.
  - Fix possible POSIX::strftime error with debug mode activated.
  - Add / at end of WebUrl when it is set but does not terminate with a slash.
  - Remove extra slash in week link, update russian translation file and fix
    some missprint. Thanks to badfiles for the patches.
  - Add Catalan translation file. Thanks to atorrillasmat for the file.
  - misprints, there are two of them. badfiles
  - Add TCP_REDIRECT to be counted as a DENIED tag from log file for users of
    squidGuard/ufdbGuard-style URL rewriters. Thanks to slashdoom for the patch.
  - Force SquidAnalyzer to use locale C internally.
  - Exclusion/Inclusion check when reading data files are limited to rebuild
    otherwise there is too much performance lost.
  - Applied exclusion/inclusion on cumulative reports even if rebuild is not
    used.
  - Fix some issue with rebuild and exclusion.
  - Show more information when a log is skipped because his size is detected
    as lower than expected.
  - Print SquidAnalyzer version when debug mode is used.
  - Add TCP_TUNNEL used by Squid 3.5 for streaming to cache miss statistics.
    Thanks to MangOuste for the report.
  - Apply exclusion/inclusion definitions on old data when rebuild is used.
    Thanks to niccarp for the feature request.
  - Fix unwanted message when QuietMode is enabled.
  - Fix typo that was crashing squid-analyzer. Thanks to Juan Jose Pablos for
    the report.
  - Fix output of benckmark info when debug is not enabled. Thanks to Juan Jose
    Pablos for the report.
  - Fix issue when rebuilding previous data without denied url stat. Thanks to
    Stepan Andreev for the report.
  - Add top denied label to translation file.
  - Add UrlHitsOnly configuration directive to be able to disable the generation
    of tables ordered per byte and duration in Top Urls and Domains report.
    Thanks to Cesar Vazquez for the feature request.
  - Add top denied url statistics. Thanks to tierpod for the featur request.
  - Replace call to localtime() to CORE::localtime() to avoid Time::localtime
    override default behaviour. Thanks to oldnrustyreaper for the report.
squidanalyzer-6.2-1.noarch [149 KiB] Changelog by Django (2015-04-30):
- Initial build for CentOS 7

Listing created by Repoview-0.6.6-4.el7