We're Moving!

Together with the release of FlowViewer version 4.0 we are moving to SourceForge

FlowViewer Version 4.2.2 includes ...

  • A quick fix for SiLK FlowTracker processing
  • Error did not impact flow-tools only users

    FlowViewer Version 4.2.2 Release Notes

    #  Version 4.2.2 is a quick emergency fix with limited changes. An array (@temp_ports)
    #  in FlowViewer_Utilities was not initialized and was acumulating contents over many
    #  FlowTracker_Collector runs. This was causing the whole FlowTracker_Collector run to
    #  slow down gradually over time as this array would have to get sorted for each SiLK
    #  FlowTracking (this caused no problems for flow-tools only users.)
    

  • FlowViewer now works with both flow-tools and SiLK at the same time making the transition to version 4 an easy one. Processing is included to carry over any filters, saved reports, or FlowTrackings established with an earlier version of FlowViewer with minor user intervention. Having a management tool that simultaneously supports older and newer netflow capture and analysis also allows network organizations to migrate to the new protocol on a schedule that is optimal for them.

    FlowViewer is an umbrella set of three tools: FlowViewer, FlowGrapher, and FlowTracker. These tools provide an easy web-based user interface for selecting, viewing, graphing, and now tracking netFlow data stored using Mark Fullmer's flow-tools (new fork) software, or, now with version 4.0, Carnegie-Mellon's NetSA group's SiLK. The user is able to filter data (inclusion or exclusion) by device, IP address range, port, router interface, autonomous system (AS), specified time interval, protocols, TOS field, TCP flags, and now Exporter, and Next Hop. Many of the flow-tools reports are configured as drop-down selections. Users are also able to save reports and graphs for later viewing, as well as track filtered data over the long run. FlowViewer makes flow data analysis and tracking quick and easy.

    The user must install and configure either flow-tools, or SiLK. Users already running FlowViewer with flow-tools may opt to install SiLK in parallel to handle IPFIX exporters while leaving older exporters in place. FlowGrapher will require the installation of Lincoln Stein's GD, and Martien Verbruggen's (now Benjamin Warfield's) GD::Graph packages for Perl. FlowTracker will require installation of Tobi Oetiker's RRDtool package. A FlowViewer configuration file is quickly configured and FlowViewer is almost immediately put into operation.

    Many thanks to FlowViewer v1-v3 users who have contributed ideas, with a special thanks to Wojciech Kozicki whose suggestion doubled the FlowViewer processing speed, and Mark Foster, Ed Ravin and others for contributions to v3. Many thanks to Carles Kishimoto, Eric Lautenschlaeger, and Sean Cardus for their ideas and code contributions to v3.3. Thanks to Dario La Guardia for pointing out a graphing problem that turned out to be a rounding error in FlowGrapher. Credit to Peter Hoffswell for the idea of linking the tools. Mike Stowe has provided valuable help with a v4.0 beta.

    Contact   Joe Loiacono   if you have problems with download, installation, configuration, or operation. Or, check the FAQ


    Requirements

    1. Web Server w/ CGI
    2. Perl v5.0 or later
    3a. flow-tools v0.68 (new fork)
    3b. flow-tools v0.67 (original)
    4. SiLK v3.0
    5. libfixbuf v1.1.2
    6. gd (for FlowGrapher)
    7. GD (for FlowGrapher)
    8. GD::Graph (for FlowGrapher)
    9. RRDtool (for FlowTracker)


    Screen Shots

    FlowViewer Main Screen
    FlowViewer Input Screen
    FlowViewer Report
    FlowGrapher Input Screen
    FlowGrapher Report
    FlowTracker Input Screen
    Creating a Group Tracking
    FlowTracker Report (partial)
    FlowTracking w/ Modified Filter
    FlowTracker Group (partial)
    Managing FlowTrackings
    Managing the Dashboard


    Download FlowViewer

    FlowViewer_4.2.2.tar
    README
    Users Guide (PDF)


    FlowViewer_4.2.1.tar
    README


    FlowViewer v4.2.2
    # README (this file) FlowViewer V4.2.2  Date: 09/28/2013
    #
    # FlowViewer is a set of three tools (FlowViewer, FlowGrapher,
    # FlowTracker) that create text reports, graph reports, and
    # long-term tracking reports from flow-tools and SiLK captured
    # and stored netflow data. FlowViewer can run with both flow-tools
    # and SiLK simultaneously. Flow-tools can handle up to v7; SilK
    # can handle v5, v9, and IPFIX.
    #
    # Software Dependencies:
    #
    #   flow-tools  http://code.google.com/p/flow-tools (If collecting v5 only)
    #   SiLK        http://tools.netsa.cert.org/silk    (If collecting IPFIX)
    #   libfixbuf   http://tools.netsa.cert.org/silk    (If collecting IPFIX)
    #   gd          http://www.libgd.org/Downloads
    #   GD          http://search.cpan.org/~lds/GD-2.30
    #   GD::Graph   http://search.cpan.org/~mverb/GDGraph-1.43
    #   GD::Text    http://search.cpan.org/~mverb/GDTextUtil-0.86/Text
    #   RRDtool     http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub
    #
    # Quick Upgrade
    #
    #   1. Untar the package into a new cgi-bin subdirectory
    #   2. Configure FlowViewer_Configuration.pm variables to your environment
    #      and create all necessary directories with proper permissions
    #   3. Replace old logos with new buttons (will be done automatically)
    #   4. Copy FlowViewer.css, FlowViewer.pdf to $reports_directory
    #   5. Configure FlowViewer_Configuration.pm to point to existing
    #      FlowTracker_Filter and FlowTracker_RRDtool directories
    #   6. Configure FlowViewer_Configuration.pm to point to existing
    #      FlowTracker html directory (or copy all existings into a new one)
    #   7. Stop old Flowtracker_Collector and FlowTracker_Grapher
    #   8. Start new Flowtracker_Collector and FlowTracker_Grapher
    #   9. Copy NamedInterfaces_Devices, names file, user logo to new directory
    #  10. Run convert_pre40_filters against existing filters
    #      (ie FlowViewer_SavedFilters)
    #  11. Use included 'User Relay' scripts if desired (recommended - see below)
    #
    # Quick Install
    #
    #   1. Untar into cgi-bin subdirectory
    #
    #   For netflow v5 and older (option):
    #
    #   2. Download, install, configure flow-tools
    #
    #   For IPFIX (e.g., v9 - also handles v5):
    #
    #   3. Download, install, configure SiLK and libfixbuf
    #
    #   For sflow (translate into v5):
    #
    #   4. Download, install, configure Inmon's sflowtool
    #
    #   For FlowViewer
    #
    #   5. Configure FlowViewer_Configuration.pm variables as necessary
    #   6. Create all necessary directories with proper permissions
    #   7. Copy FlowViewer.css, FlowViewer.pdf to $reports_directory
    #   8. Point browser to FV.cgi
    #
    #   For FlowGrapher
    #
    #   9. Install gd (C), GD (Perl), GD::Graph (Perl) GD::Text (Perl)
    #  10. Configure FlowViewer_Configuration.pm variables as necessary
    #  11. Point browser to FV.cgi
    #
    #   For FlowTracker
    #
    #  12. Install RRDtool (at least version 1.4)
    #  13. Create FlowTracker_Filter and FlowTracker_RRDtool directories
    #  14. Configure FlowViewer_Configuration.pm variables as necessary
    #  15. Start FlowTracker_Collector, FlowTracker_Grapher in background
    #  16. Point browser to FV.cgi
    #
    #   For all FlowViewer tools
    #
    #  17. Review all FlowViewer directories and files for proper permissions
    #
    # Version 4.2.2 Release Notes
    #
    #  Version 4.2.2 is a quick emergency fix with limited changes. An
    #  array (@temp_ports) in FlowViewer_Utilities was not initialized
    #  and was acumulating contents over many FlowTracker_Collector runs.
    #  This was causing the whole FlowTracker_Collector run to slow down
    #  gradually over time as this array would have to get sorted for each
    #  SiLK FlowTracking (this caused no problems for flow-tools only
    #  users.)
    #
    # Version 4.2.1 Release Notes
    #
    #  Version 4.2.1 extends the use of the recently discovered flow-report
    #  option "linear-interpolated-flows-octets-packets" to FlowTracker
    #  processing (see User's Guide for further discussion.) This provides
    #  significant speed-ups for FlowTracker_Recreate (above 75%) and
    #  FlowTracker_Collector (which will now permit many more FlowTrackings.)
    #  The FlowTracker_Recreate processing for SiLK was remarkably poorly
    #  implemented and this has been corrected introducing unspeakable
    #  performance gains there as well. The same use of SiLK prefiltering
    #  that was introduced for FlowTracker in v4.2 has been applied to
    #  FlowGrapher for a speed-up there. A new capability is added to work
    #  with different international date formats. The distribution includes a
    #  new capability, FlowViewer_CleanSiLK, to monitor and adjust diskspace
    #  used by IPFIX devices addressing the SiLK deficiency of not having the
    #  valuable flow-capture feature (-E) of active diskspace usage control.
    #  Made the use of prefiltered files or CONCATs non-optional and fixed
    #  SiLK processing of flows and packets for FlowTracker_Collector. Modest
    #  changes have been made to FlowViewer_CleanFiles, FV.cgi and
    #  flowtracker_grapher_nonlazy. An error causing packets to be tracked as
    #  flows in some cases was fixed. A new tool, flowtracker_grapher_recent
    #  will re-graph (nonlazy) only recently created FlowTrackings.
    #
    # Version 4.2 Release Notes
    #
    #  Version 4.2 incorporates the equivalent of "use existing concatenations"
    #  for SiLK based FlowTrackings. FlowViewer takes advantage of previously
    #  concatenated flow-tools files during FlowTracker_Collector processing
    #  to speed up the whole run. Now it does this for SiLK files as well by
    #  performing rwfilter "INPUT" filtering only once for all FlowTrackings
    #  that are based on the same sensor/class combination. This is a pretty
    #  significant speed-up and will permit the user to have many more
    #  FlowTrackings. For example, our implementation, a combination of
    #  flow-tools and SiLK based data, now processes 250 FlowTrackings in 35
    #  seconds (prior to the new version this was taking 50 seconds.) The new
    #  version also corrects processing of exporters [M. Donnelly]. Excluded
    #  fields (e.g., protocols=-17) were being accepted for SiLK FlowTrackings
    #  despite SiLK not being able to handle them; this was fixed. The
    #  analyze_netflow_packets tool has been fixed for IPv6 addresses. Four new
    #  date conversion utilities have been added to the 'tools' subdirectory.
    #  FlowTracker_Collector was modified in version 4.0 to be able to create
    #  FlowTrackings for flows and packets (as well as bits.) This was
    #  inadvertantly dropped in version 4.1 and is restored in this update
    #  to version 4.2 [7/31].
    #
    # Version 4.1 Release Notes
    #
    #  Version 4.1 includes a new FlowGrapher capability that creates reports
    #  3 to 4 times faster than previously. The detail lines are a little bit
    #  different. The previous capability is retained offering the user a choice
    #  from the input form interface. The new FlowGrapher report type is
    #  "Aggregated". It makes use of a heretofore missed flow-tools capability
    #  known as the flow-report "linear-interpolated-flows-octets-packets" option
    #  which aggregates flows, octets, or packets into time buckets. This moves
    #  that processing into the compiled "C" code of flow-tools. New FlowTracker
    #  capabilities are added to track flows or packets as well as the previously
    #  available octets. Version 4.1 introduces the ability to maintain different
    #  dashboards for different users (please see the User's Guide for how to do
    #  this. The new version includes a new FV_Relay.cgi script. The new version
    #  fixes a flaw in FlowTracker_Collector that erroneously tracked protocols,
    #  tcp_flags, and tos_fields when using SiLK (thanks C. Spitzlay.) It also
    #  includes some small fixes like making directory creation a little easier,
    #  fixes removing (and adding) Trackings from the Dashboard and removes some
    #  minor extraneous formatting. Fixed problem with FlowTracker_Collector
    #  processing of SiLK interface filtering [07/09/13]. Fixed initialization
    #  of @ipfix_devices in FlowViewer_Configuration.pm [Thanks M. Donnelly.]
    #
    # Version 4.0 Release Notes
    #
    #  Version 4.0 is a major upgrade that enables FlowViewer to handle IPFIX
    #  netflow data (i.e., v9, etc.) The User Interface has been completely redone
    #  and now features a Dashboard. Aside from the new collector interface and user
    #  interface, version 4.0 introduces some new capabilities:
    #
    #   1. FlowViewer report sorting by column header
    #   2. Dashboard of thumbnail versions of selected FlowTracker graphs
    #   3. Ability to 'recreate' FlowTrackings, starting at a time specified in the
    #      past
    #
    #  The distribution manifest has changed significantly.
    #
    #  Preserved Scripts, Files, and Tools:
    #
    #   FlowViewer.cgi                Modified for new user interface.
    #   FlowViewer_Main.cgi           Modified for new user interface.
    #   FlowViewer_Relay.cgi          No change.
    #   FlowViewer_Save.cgi           Significant modification.
    #   FlowGrapher.cgi               Modified for new user interface.
    #   FlowGrapher_Main.cgi          Modified for new interface.
    #   FlowGrapher_Colors            No change.
    #   FlowGrapher_Relay.cgi         No change.
    #   FlowGrapher_Sort.cgi          Significant modification.
    #   FlowTracker.cgi               Modified for new user interface.
    #   FlowTracker_Collector         Modified to process stored SiLK data.
    #   FlowTracker_Grapher           Modified to update Thumbnails.
    #   FlowTracker_Group             Modified for new user interface.
    #   FlowTracker_Dumper            Modified for new user interface.
    #   FlowTracker_Relay.cgi         No change.
    #   FlowViewer_CleanASCache       No change.
    #   FlowViewer_CleanFiles         Minor changes.
    #   FlowViewer_CleanHostCache     No change.
    #   FlowViewer_Configuration.pm   Modifications for SiLK and user interface.
    #   FlowViewer_Utilities.pm       Removed filter output processing.
    #   NamedInterfaces_Devices       No change.
    #   NamedInterfaces_Exporters     No change.
    #   flowcapture_restart           No change.
    #   flow-capture-table.conf       No change.
    #   flowtracker_restart           No change.
    #   performance_check             Parse FlowTracker logs and report performance
    #   rsync_flows                   Rsync all of raw flow data to backup host
    #   rsync_trackings               Rsync all of Tracking data to backup host
    #
    #  New Scripts, Files, and Tools
    #
    #   FlowViewer_Replay.cgi         Presents saved FlowViewer reports
    #   FlowViewer_SaveManage.cgi     Manages saved reports
    #   FlowViewer_Sort.cgi           Sorts FlowViewer reports
    #   FlowViewer_UI.cgi             Utilities for creating user interface
    #   FlowGrapher_Replay.cgi        Presents saved FlowGrapher reports
    #   FlowTracker_Dashboard.cgi     Manages the Dashboard contents
    #   FlowTracker_Display.cgi       Presents a FlowTracking
    #   FlowTracker_DisplayPublic.cgi Presents a FlowTracking from Public list
    #   FlowTracker_Management.cgi    Manages FlowTrackings (e.g., remove, etc.)
    #   FlowTracker_Recreate          Background process to recreate FlowTrackings
    #   FlowTracker_Thumbnail         Invoked to create a Thumbnail FlowTracking
    #   FlowViewer.css                FlowViewer cascading style sheet
    #   FV_button.png                 New button link to FlowViewer from front page
    #   FG_button.png                 New button link to FlowGrapher from front page
    #   FT_button.png                 New button link to FlowTracker from front page
    #   convert_pre40_filters         Converts old saved filters (pre version 4.0).
    #   flowtracker_archive_restore   Restores archived FlowTrackings gone astray
    #   flowtracker_grapher_nonlazy   Forces a re-graphing of all FlowTracker graphs
    #   resize_rrdtools               Extends RRDtools created prior to 3-Year graph
    #   rwflowpack_start              One-line script starts SiLK collector
    #   analyze_netflow_packets       Script analyzes TCPDUMP captured netflow data
    #
    #  Removed Scripts and Files
    #
    #   FlowViewer_SavedFilters       File kept saved filters
    #
    #  General Notes:
    #
    #   Remember to copy into the new directory (e.g.,
    #   /usr/lib/cgi-bin/FlowViewer_3.4) user logos, names file, as_names,
    #   NamedInterfaces_Devices, NamedInterface_Exporters, FlowViewer_SavedFilters,
    #   etc., from the old cgi-bin directory.
    #
    #   The simplest way to transition to the new version is to leave all
    #   FlowViewer_Configuration.pm settings alone except:
    #
    #    $reports_directory       = "/var/www/FlowViewer_3.4";
    #    $reports_short           = "/FlowViewer_3.4";
    #    $graphs_directory        = "/var/www/FlowGrapher_3.4";
    #    $graphs_short            = "/FlowGrapher_3.4";
    #    $tracker_directory       = "/var/www/FlowTracker_3.4";
    #    $tracker_short           = "/FlowTracker_3.4";
    #    $old_tracker_directory   = "/var/www/FlowTracker_3.3.1";
    #    $cgi_bin_directory       = "/usr/lib/cgi-bin/FlowViewer_3.4";
    #    $cgi_bin_short           = "/cgi-bin/FlowViewer_3.4";
    #    $work_directory          = "/usr/lib/cgi-bin/FlowViewer_3.4/Flow_Working";
    #    $work_short              = "/cgi-bin/FlowViewer_3.4/Flow_Working";
    #    $names_directory         = "/usr/lib/cgi-bin/FlowViewer_3.4";
    #    $log_directory           = "/usr/lib/cgi-bin/FlowViewer_3.4"
    #
    #   The following can remain the same (or copy contents to the new directory):
    #
    #    $save_directory          = "/var/www/FlowViewer_Saves";
    #    $save_short              = "/FlowViewer_Saves";
    #    $filter_directory        = ".../FlowTracker_Files/FlowTracker_Filters";
    #    $rrdtool_directory       = ".../FlowTracker_Files/FlowTracker_RRDtool";
    #
    #   If this is an upgrade for you (e.g., from v3.3.1) I recommend using the
    #   FlowViewer_Relay.cgi, FlowGrapher_Relay.cgi, and the FlowTracker_Relay.cgi
    #   scripts to alert users to the new version with links and a reminder to
    #   change their bookmarks. In each of the relay scripts tailor the following
    #   line to your environment (point to the new FlowViewer_Configuration.pm file):
    #
    #    require "/usr/lib/cgi-bin/FlowViewer_3.4/FlowViewer_Configuration.pm";
    #
    #   ... then, in your old cgi-bin directory (e.g., FlowViewer_3.3.1), copy the
    #   following:
    #
    #    cp FlowViewer_Relay.cgi FlowViewer.cgi
    #    cp FlowGrapher_Relay.cgi FlowGrapher.cgi
    #    cp FlowTracker_Relay.cgi FlowTracker.cgi
    #
    #   Now, when users go to their book-marked FlowViewer web page, they will be
    #   directed to the new one. FlowTracker_Relay.cgi is particularly important
    #   this is an upgrade it copies over archived FlowTrackings which would be a
    #   bit tedious to copy by hand.
    #
    #   The rsync_flows and rsync_trackings scripts are useful for easily backing
    #   up all raw netflow data and FlowTracker state information (Filters and
    #   RRDtool databases.) The FlowViewer_CleanFiles script is useful for deleting
    #   aging files that are not necessary anymore. I run it out of 'cron' once a
    #   day.
    #
    #   The performance_check script can be used from the command line to keep track
    #   of how well your implementation is performing. I run it against my
    #   Flowtracker_Collector.log file to see how things are going. Here at the
    #   NASA Earth Observing System network I have over 200 FlowTrackings and
    #   they complete in an average of 44 seconds. FlowTracker_Collector runs every
    #   five minutes and I watch for runs that take longer than five minutes.
    #   Even in those situations, however, FlowTracker_Collector seems to continue
    #   on with no real visible effects.
    #
    # Version 3.3 Release Notes
    #
    # ### Version 3.3.1 fixes a FlowTracker_Collector bug when using exporters
    #     Also fixes problem for users without devices at all. If you are not
    #     using any devices (or exporters) you will now have to set:
    #     $no_devices_or_exporters = "Y"; Fixes FlowGrapher sorting of host
    #     names. This version fixes the problem of links to Trackings embedded
    #     in Group graphs not lining up properly. Fixes problem with
    #     FlowTracker_Grapher not printing out named interfaces. Fixes
    #     FlowGrapher graph and output to now have exporter name. Fixes
    #     problem with end-of-month graphs (missing days_in_month.)
    #
    # New Capabilities
    #
    #   1. Some devices will now have 'named interfaces' (thanks C. Kishimoto)
    #   2. The user can now save filters of interest and recall them later
    #   3. Data can now be analyzed by Exporter ID (in addition to device name)
    #   4. Users can now set thresholds on FlowTrackings, and be alerted
    #   5. Users can now sort FlowGrapher output based on column type
    #   6. FlowViewer now provides Pie Charts
    #   7. Capability added to apply a Sampling Multiplier to output
    #   8. FlowTrackings now have a '3 year' graph
    #   9. The user can now generate text listings of FlowTracker output
    #  10. Filtering on next-hop has been added
    #  11. Logging has been made more flexible (e.g., less data)
    #  12. Preserve latest three notations (was keeping first three)
    #  13. Can now specify and display time-zones
    #  14. A hook has been provided for a User Logo with link out of FlowViewer
    #  15. New file cleanup scripts have been added
    #  16. Unit Conversion capability has been added (thanks C. Kishimoto)
    #  17. Can now graph Flows, Packets as well as Octets (thanks E.Lautenschlaeger)
    #  18. Improved AS name resolution (thanks S. Cardus)
    #  19. New saved_directory for storing saved Reports and Graphs.
    #
    # New Scripts and Files:
    #
    #  FlowGrapher_Sort.cgi        Sorts FlowGrapher Detail Lines by column
    #  FlowTracker_Dumper.cgi      Invoked by link in Trackings, prints text values
    #  FlowViewer_Save.png         New logo with links for saving filters, reports
    #  Flowgrapher_Save.png        New logo with links for saving filters, reports
    #  FlowViewer_CleanASCache     Tool used to remove obsolete AS name resolutions
    #  FlowViewer_CleanFiles       Tool used to remove old intermediate files
    #  FlowViewer_CleanHostCache   Tool used to remove obsolete host name resolutions
    #  FlowViewer_Relay.cgi        Optional: points users to new version (see Notes)
    #  FlowGrapher_Relay.cgi       Optional: points users to new version (see Notes)
    #  FlowTracker_Relay.cgi       Optional: points users to new version (see Notes)
    #  flowcap                     Optional start-up script for flow-tools and
    #                              FlowTracker
    #
    #  NamedInterfaces_Devices     Holds interface names for SNMP device indices
    #  NamedInterfaces_Exporters   Holds interface names for SNMP index (exporters)
    #  FlowViewer_SavedFilters     Created during processing to hold saved filters
    #
    # Notes:
    #
    #  Many thanks to Carles Kishimoto, Eric Lautenschlaeger, and Sean Cardus for
    #  their ideas and code contributions. Thanks to Dario La Guardia for pointing
    #  out a graphing problem that turned out to be a rounding error in FlowGrapher.
    #  Credit to Peter Hoffswell for the idea of linking the tools.
    #
    #  There are no new software dependencies with FlowViewer version 3.3.1, however
    #  Named Interfaces now requires Javascript in the browser to operate.
    #
    #  If you are having trouble with creating Tracking Groups, you may have a
    #  problem with the installation of RRDs.pm. This needs to be placed in a
    #  library that Perl includes in it's @INC array. For a fix, see the FlowViewer
    #  FAQ on the web site.
    #
    #  Using the 'Relay' scripts (these are optional)
    #
    #   If you have other users and you would like to point them to the new version,
    #   copy the included 'Relay' scripts over the old FlowViewer.cgi,
    #   FlowGrapher.cgi and FlowTracker.cgi scripts in the last version's directory.
    #
    #   For example:
    #
    #    In the old directory /htp/cgi-bin/FlowViewer_3.2:
    #
    #    mv FlowViewer_Relay.cgi  FlowViewer.cgi
    #    mv FlowGrapher_Relay.cgi FlowGrapher.cgi
    #    mv FlowTracker_Relay.cgi FlowTracker.cgi
    #
    #   Then, when the user goes to the old FlowViewer, he will be provided a link
    #   to the new FlowViewer, and asked to change his bookmarks.
    #
    #
    #  Setting up crontab file for cleaning FlowViewer files:
    #
    #   min   hr  dom  moy  dow    command
    #
    #     5   0    *    *    *     .../FlowViewer_3.4/FlowViewer_CleanFiles
    #                                > .../FlowViewer_3.4/cleanup.log
    #                                2 >> .../FlowViewer_3.4/cleanup.log
    #
    #  The file cleanup is controlled by parameters in FlowViewer_Configuration.pm:
    #
    #   $remove_workfiles_time   = 86400;
    #   $remove_graphfiles_time  = 7*86400;
    #   $remove_reportfiles_time = 7*86400;
    #
    #  Remember, whichever crontab account this is started from must have adequate
    #  permissions to remove files created by the web process owner (e.g., apache.)
    #
    # Documentation
    #
    # The FlowViewer User's Guide is available on the FlowViewer Website:
    #
    #   http://ensight.eos.nasa.gov/FlowViewer
    #
    # Dependencies
    #
    # - FlowGrapher requires the Perl GD and GD:Graph packages
    #   gd package. Thomas Boutrell's graphics package written in 'C'
    #   GD package: http://search.cpan.org/~lds/GD-2.30/
    #   GD::Graph package: http://search.cpan.org/~mverb/GDGraph-1.43/
    # - FlowViewer.cgi requires the GDBM or NDBM capability in Perl
    # - FlowTracker requires RRDtool (at least version 1.2.12)
    #   RRDtool: http://oss.oetiker.ch/rrdtool
    #
    # Contents
    #
    # FlowViewer_Configuration.pm
    #
    # This file contains parameters that configure and control the
    # FlowViewer, FlowGrapher, and FlowTracker environments. This package
    # should remain in the same directory that the CGI scripts are in.
    #
    # FlowViewer_Utilities.pm
    #
    # This file contains processing used by multiple programs (e.g., to
    # create the Report Parameters output for each tool, and other utilities
    # (e.g., 'epoch_to_date' which converts between typical date formats
    # and 'seconds since 1972') that are invoked by other scripts. This
    # package should be placed in the same directory as the CGI scripts.
    #
    # FlowViewer.cgi
    #
    # This script produces the web page which provides the user the form
    # for entering analysis selection criteria for FlowViewer. Version 3.0
    # reorganized the processing. FlowViewer.cgi is now the old
    # create_FlowViewer_webpage. This change permits the input date and time
    # to be updated with each invocation.
    #
    # FlowViewer_Main.cgi
    #
    # This script responds when the user completes the selection criteria
    # form and submits the 'Generate Report' command. The script creates a
    # flow-tools filter file based on the selection criteria. Based on the
    # input time period, the script concatenates the relevant flow-tools
    # data files for the selected device. The location of the flow-tools
    # raw data files is specified via the 'flow_data_directory' parameter.
    # The script then invokes the selected statistics/print report flow-tools
    # program and reformats the output into HTML. An option is available in
    # FlowViewer_Configuration to have this script use the NDBM capability
    # (for caching resolved host names) instead of the default GDBM
    # capability for users whose Perl distribution does not have GDBM.
    #
    # FlowGrapher.cgi
    #
    # This script produces the web page which provides the user the form
    # for entering analysis selection criteria for FlowGrapher. Version 3.0
    # reorganized the processing. FlowGrapher.cgi is now the old
    # create_FlowGrapher. This change permits the input date and time
    # to be updated with each invocation.
    #
    # FlowGrapher_Main.cgi
    #
    # This script responds when the user completes the FlowGrapher selection
    # criteria form and submits the 'Generate Graph' command. The script
    # creates intermediate processing files exactly like FlowViewer above.
    # The script then parses intermediate output, fills time buckets, and
    # generates a graphic image. Textual output accompanies the graph. An
    # option is available in FlowViewer_Configuration to have this script use
    # the NDBM capability (for caching resolved host names) instead of the
    # default GDBM capability for users whose Perl distribution does not have
    # GDBM.
    #
    # FlowGrapher_Sort.cgi
    #
    # This script is invoked when the user clicks on a column header for the
    # Detail Lines of a FlowGrapher report. The textual data on the page is
    # sorted and re-presented.
    #
    # FlowGrapher_Colors
    #
    # This file contains a translation between textual color names and their
    # RGB value counterparts.
    #
    # FlowTracker.cgi
    #
    # This script produces the web page which provides the user the form
    # for entering analysis selection criteria for FlowTracker. The script
    # also provides the user with the ability to review, revise, or remove
    # existing trackings. FlowTracker was new in version 3.0.
    #
    # FlowTracker_Main.cgi
    #
    # This script responds when the user completes the FlowTracker selection
    # criteria form and submits the 'Establish Tracking' command. The script
    # responds to the users desire to create, remove, or revise a tracking.
    #
    # FlowTracker_Group.cgi
    #
    # This script controls the building of groups from existing Individual
    # FlowTrackings. The user has the ability with FlowTracker v3.2 to create
    # 'groups' from pre-defined Individual trackings. A Group Tracking has no
    # RRD database associated with it, but simply creates a multifaceted graph
    # from several existing trackings. The Group 'merges' the Individual graphs
    # onto a single graph.
    #
    # FlowTracker_Dumper.cgi
    #
    # This script is invoked when the user clicks on a link within the
    # FlowTracking graph labeled '[List values]'. The script dumps the
    # RRDtool contents onto a web page.
    #
    # FlowTracker_Collector
    #
    # The script is started once by the user and placed in the 'background'.
    # The script will execute and then sleep for the duration of a five minute
    # period, essentially running every five minutes. For each existing tracking,
    # the script applies the associated filter to the flow data and extracts the
    # amount that occured during a 5-minute window approximately 30 miuntes
    # earlier. This is to permit long-running flows to have been exported and
    # available to the collector. The script then divides the total bits by
    # 300 seconds to get an average bits-per-second rate during the period.
    # The data point is then provided to RRDtool for storage. The script
    # should be started out of the cgi-bin directory.
    #
    # FlowTracker_Grapher
    #
    # The script is started once by the user and placed in the 'background'.
    # The script will execute and then sleep for the duration of a five minute
    # period, essentially running every five minutes. The script runs the
    # RRDtool graph function for each existing tracking. Daily, weekly,
    # monthly, and yearly graphs are updated with the latest information. The
    # script creates an html page for each tracking that includes the filter
    # parameters and the four graphs. The script also creates an overall web
    # page ($tracker_webpage) that provides links to all active tracking pages.
    # The script should be started out of the cgi-bin directory.
    #
    #
    # FlowViewerM.png
    #
    # The FlowViewerM logo with links. Leave this file in the 'cgi-bin_directory',
    # the FlowViewerM.cgi script will place a copy of the image in
    # 'html_directory'. This image contains mapped links to FlowGrapher and
    # FlowTracker such that those input pages are pre-loaded with the filter
    # criteria from FlowViewer.
    #
    # FlowViewerS.png
    #
    # The FlowViewerS logo with links. Leave this file in the
    # 'cgi-bin_directory', the FlowViewerS.cgi script will place a copy of the
    # image in 'reports_directory'. This image contains mapped links to the other
    # tools as well as links for saving the filter used or the report generated.
    #
    # FlowGrapherM.png
    #
    # The FlowGrapherM logo with links. Leave this file in the 'cgi-bin_directory',
    # the FlowGrapherM.cgi script will place a copy of the image in
    # 'graphs_directory'. This image contains mapped links to FlowViewer and
    # FlowTracker such that those input pages are pre-loaded with the filter
    # criteria from FlowGrapher.
    #
    # FlowGrapherS.png
    #
    # The FlowGrapherS logo with links. Leave this file in the
    # 'cgi-bin_directory', the FlowGrapherS.cgi script will place a copy of the
    # image in 'graphs_directory'. This image contains mapped links to the other
    # tools as well as links for saving the filter used or the report generated.
    #
    # FlowTrackerM.png
    #
    # The FlowTracker logo with links. Leave this file in the 'cgi-bin_directory',
    # the FlowTrackerM.cgi script will place a copy of the image in
    # 'tracker_directory'. This image contains mapped links to FlowViewer and
    # FlowGrapher such that those input pages are pre-loaded with the filter
    # criteria from FlowTracker.
    #
    # FlowViewer_Save.cgi
    #
    # This script moves temporary save files into a permanent residence
    # as defined by either the 'reports_directory' or 'graphs_directory'
    # environment variables.
    #
    # FlowViewer_CleanFiles
    #
    #  A utility for cleaning out temporary files that have been left
    #  over from debugging (e.g. $debug_files = 'Y'). Files older than
    #  the following configurable parameters are removed:
    #
    #   $remove_workfiles_time   = 86400;
    #   $remove_graphfiles_time  = 7*86400;
    #   $remove_reportfiles_time = 7*86400
    #
    #  See above for crontab settings for running this automatically.
    #
    # FlowViewer_CleanASCache
    #
    # A utility for cleaning out from the AS resolving cache ($as_file) a
    # resolved AS name that is no longer valid.
    #
    # FlowViewer_CleanHostCache
    #
    # A utility for cleaning out from the DNS resolving cache ($names_file)
    # a resolved host name that is no longer valid.
    #
    # FlowViewer_Relay.cgi, FlowGrapher_Relay.cgi, FlowTracker_Relay.cgi
    #
    # Short scripts that refer users from version 3.3.1 to version 3.4. This
    # keeps you from having to notify users to go to a different web site.
    #
    # flowcapture_restart
    #
    # A shell script used for starting up and restarting flow-captures. Tailor
    # this for your environment.
    #
    # flowtracker_restart
    #
    # A shell script used for starting up and restarting FlowTracking_Collector
    # and FlowTracker_Grapher. Tailor this for your environment.
    #
    # Generic_Logo.jpg
    #
    # This image is to be replaced by your own image that can point back to
    # anywhere (e.g., your overarching NMS system.)
    #
    # NamedInterfaces_Devices
    #
    # This file is used for SNMP index to named interface translation. This file
    # provides translation when you are saving data by individual devices. Examples
    # are provided.
    #
    # NamedInterfaces_Exporters
    #
    # This file is used for SNMP index to named interface translation. This file
    # provides translation when you are saving data into a single directory but
    # for (possibly) multiple devices differentiated by EXPORTER_ID. Examples are
    # provided.
    #
    # Configuration parameters
    #
    # The FlowViewer, FlowGrapher, and FlowTracker scripts all use parameters
    # in the FlowViewer_Configuration.pm file to control the environment that
    # they run in. Here is a brief explanation of some of the relevant
    # parameters:
    #
    # $ENV(PATH) - modify as appropriate for your installation
    # $FlowViewer_server - IP address of server hosting this software
    # $FlowViewer_service - Either HTTP (port 80) or HTTPS (port 443)
    # $reports_directory - Directory to hold saved FlowViewer reports
    # $reports_short - Reports directory beginning from web server default
    # $graphs_directory - Directory to hold saved FlowGrapher reports
    # $graphs_short - Graphs directory beginning from web server default
    # $tracker_directory - Directory to hold FlowTracker trackings
    # $tracker_short - Tracker directory beginning from web server default
    # $filter_directory - Directory in which to keep FlowTracker filter files
    # $rrdtool_directory - Directory in which to keep FlowTracker RRDtool files
    # $cgi_bin_directory - Directory which holds cgi scripts
    # $cgi_bin_short - cgi-bin directory beginning from web server default
    # $flow_data_directory - Directory that holds all flow-tools data files
    # $exporter_directory - Directory where netflow stored for multiple exporters
    # $flow_bin_directory - Directory where all flow-tools reside
    # $rrdtool_bin_directory - Location of RRDtool programs
    # $work_directory - Directory to store intermediate files
    # $names_directory - Directory to save permanent 'names' file
    # $flow_capture_interval - Interval beyond end point to capture all flows
    # $flow_file_length - Length (in seconds) of each of your flow files
    # $devices - List of device names exporting netflow (see #4 below)
    # $no_devices_or_exporters - Set to "Y" if you have no devices and no exporters
    # $N - Used to control directory organization (see #5 below)
    # $dig - Location of DNS utility 'dig' (set to nslookup if required)
    # $actives_webpage - Name of HTML file which will list your Trackings
    # $trackings_title - Title for HTML page which lists Trackings
    # $user_logo - Filename of image used for your logo
    # $user_hyperlink - Link associated with $user_logo
    # $use_even_hours - Will start default time periods at the top of the hour
    # $use_NDBM - Some Perls don't have GDBM (default), but do have NDBM
    # $start_offset - Offset from current time for beginning pre-loaded time period
    # $end_offset - Offset from current time for end of pre-loaded time period
    # $flow_capture_interval - Minutes beyond end period for collecting all flows
    # $flow_file_length - Size (minutes) of each flow-tools flow file (default = 15)
    # $labels_in_titles - Whether to print FlowTracker title in the graph itself
    # $debug_files - If Yes, will not remove intermediate files
    # $collection_offset - Seconds into past to begin collection period
    # $collection_period - Period to examine for FlowTracker (keep at 5 minutes!)
    # $use_existing_concats - DEPRECATED. Re-use concatenations (much faster)
    # $rrd_dir_perms - (And others) UNIX directory or file permissions
    #
    # The rest of this file contains basic parameters such as colors, etc. Each
    # parameter is dicussed in more detail in the User's Guide.
    #
    # Additional Considerations
    #
    # 1. Directory permissions for the subdirectories created for the
    # 'htdocs', 'work', 'names', 'cgi-bin' (e.g., FlowTracker_Filter,
    # FlowTracker_RRDtool) directories must permit the owner of the web
    # server process (e.g., apache) to write into these directories.
    # The directories may have been created by a different user. Version 3.0
    # introduced the use of $dir_perms. There are several of these included
    # in FlowViewer_Configuration.pm. These are the permissions that the
    # scripts will set your various FlowViewer files and directories to.
    # They default to '0777' which permits the open interaction between the
    # web server process owner and the FlowTracker background process owner.
    # You may want to adjust these permissions differently according
    # to your security policies, and whether you use the same or different
    # accounts for the web and background processes.
    #
    # 2. FlowViewer and FlowGrapher offer the ability to save interesting
    # reports. To do this, the scripts save a temporary copy of every report
    # in advance of the user electing to save it permanently. These
    # intermediate files will accumulate in the 'work' directory specified
    # in the FlowViewer_Configuration file. These files could be removed
    # daily via a cron script to prevent unecessary use of disk space. When
    # the user elects to save a report, it is copied into either the
    # 'reports_directory', or the 'graphs_directory' depending on which
    # function he is running. See discussion of FlowViewer_CleanFiles above.
    #
    # 3. FlowViewer and FlowGrapher offer the ability to resolve NetFlow IP
    # addresses into their host names on the fly. This process is speeded
    # up by caching names into a 'names' file which resides in the directory
    # specified by the 'names_directory' parameter. This parameter defaults
    # to /tmp, but this may not be the best directory for you since it will
    # disappear with a reboot. As you are building up your 'names' file
    # with early runs, you will notice the speed increase dramatically
    # as the 'names' file is used more. The process of resolving names is
    # the primary reason for slower overall FlowViewer performance. You
    # should preferably use the GDBM array database which is fastest.
    # However, not all Perl distributions support GDBM but most do support
    # NDBM. The '$use_NDBM' flag in FlowViewer_Configuration.pm will
    # cause the FlowViewer_Main and FlowGrapher_Main scripts to use NDBM.
    #
    # 4. The FlowViewer and FlowGrapher reporting features use a flow-tools
    # data directory layout that has a particular device at the top. A
    # typical flow-tools directory looks like:
    #
    # /flows/router_1/2005/2005-07/2005-07-04
    #
    # The device name (router_1) is obtained from an array called 'devices'
    # in the FlowViewer_Configuration.pm file. Populate this array with your
    # device names. If your flow-data file structure does not include a
    # device name, for example you are collecting only from one device, set
    # the @devices array to empty (i.e., @devices = ("");) On the web page
    # you can ignore the Devices pulldown selection.
    #
    # As of version 3.3, users may now apply FlowViewer to directories that
    # collect from multiple sources, differentiated by EXPORTER_ID. If you
    # are using this method (i.e., all flow-captures going into a single
    # directory), simply set the $exporter_directory parameter to the
    # directory that is set up to store the flow-data files.
    #
    # 5. Different organizations store captured netflow data differently
    # according to the 'N" setting on the flow-capture statement. However,
    # there is a bug in the flow-tools documentation such that the default
    # value is truly '3' and not '0' as indicated. I have set $N = 3 to
    # reflect the more common setting. The directory structure associated
    # with $N = 3 is shown below:
    #
    # /flows/router_1/2005/2005-07/2005-07-04
    #
    # If you are not seeing output, please check this setting.
    #
    # 6. Version 3.2 introduces Groups. Intermediate RRDtool databases are
    # created on the fly in order to create a temporary FlowTracker graph that
    # shows the user how the final graph will look. It uses the Perl RRDs.pm
    # RRDtool module to speed this up. Make sure your RRDtool distribution
    # has a compatible RRDs.pm module.
    #
    # Change Log
    #
    # Version 3.4 - March 17, 2011
    #
    # See Version 3.4 Release Notes above
    #
    # Version 3.3
    #
    # See Version 3.3 Release Notes above
    #
    # Version 3.2
    #
    # Version 3.2 introduces Group trackings which are simply a tracking graph
    # made up from the merging of several predefined Individual trackings onto
    # one graph. There are no permanent RRDtool databases associated with a Group.
    # In the construction of a group however, temporary RRDtool databases are
    # created to simulate how the Group will eventually look. The new script
    # Flowtracker_Group uses RRDs.pm (comes with RRDtool) to generate these
    # transient databases quickly. Please ensure that your RRDs.pm module is
    # compatible with your RRDtool distribution (this should normally be the
    # case - but if you see "ERR: can't handle RRD file version 0003" in
    # DEBUG_GRAPHER, you'll need to upgrade your RRDs.pm.
    #
    # This version also includes a 'speed-up' for FlowTracker_Collector which
    # now concatenates once for each device. This is controlled by
    # $use_existing_concats, which defaults to "Y".
    #
    # Version 3.2 will continue to work happily along with earlier version
    # exiting Filter and RRDtool files. There are no new FlowViewer_Configuration
    # parameters of consequence. There are now two types of trackings; Individual
    # and Group. The FlowTracker input screen will default to Individual which is
    # the same as the existing trackings.
    #
    # Woj Kozicki has contributed an Autonomous System (AS) resolving capability
    # and it is included in v 3.2.
    #
    # New FlowViewer, FlowGrapher, and FlowTracker logos have been developed for
    # version 3.2. These new logos provide embedded links to the other tools so that
    # the user can switch between them easily and retain input parameters.
    #
    # Version 3.1
    #
    # 1. Added MIN, MAX, AVG, 95th PCT to FlowGrapher
    # 2. Added ability to 'archive' trackings
    # 3. Added ability to enter port ranges separated by a colon (:)
    # 4. Can now use any mask length for networks (1 - 32)
    # 5. Added RRDgraph 'lazy-mode' option to speed up graphing
    # 6. Upgraded FlowViewer/Grapher ability to go back more than 30 days
    # 7. Added the ability to configure file permissions
    # 8. Improved speed of FlowGrapher for larger values of 'detail lines'
    # 9. Fixed $rrdtool_bin-directory variable name
    # 10. Added ability to retain intermediate files for debugging
    # 11. Sorted list of Active Trackings
    # 12. Fixed bug where non-zero 'cutoof lines' would supress some reports
    # 13. Fixed FlowViewer rate output to calculate average from all flows
    # 14. Fixed FlowTracker_Collector log output (to collect_period_average)
    #
    # Version 3.0
    #
    # 1. Major new addition of FlowTracker
    # 2. Reorganized scripts so that the date and time fields are updated
    #    with each invocation
    # 3. Moved common code (e.g., filter creation) to FlowViewer_Utilities.pm
    # 4. Improved Report Parameters output formatting
    # 5. Provided host names capability for FlowGrapher (thanks Mark Foster)
    # 6. Introduced debug and logging capabilities
    # 7. Merged GDBM/NDBM into a single script (thanks Ed Ravin)
    #
    # Version 2.3
    #
    # 1. Modified FlowGrapher record processing to not call 'timelocal' for
    #    epoch times. Other speed improvements. Result: up to 10 times faster.
    # 2. FlowGrapher error leaving spikes is fixed (thanks Mark Foster)
    # 3. Bug with concatenation when $N=0 fixed (thanks Dave Faught)
    #
    # Version 2.2
    #
    # 1. Added flow_select parameter to control which flows are considered
    #    with respect to the specified time period
    # 2. Removed Easterm Time (ET) notation. All times are system local
    #
    # Version 2.1
    #
    # 1. Fixed concatenation. Needs to start one flow file length before start time
    # 2. Fixed end-of-year problem in FlowGrapher
    # 3. Small problem for time requests that end just before midnight
    #
    # Version 2.0
    #
    # 1. Used pipe (|) instead of reading intermediate files (thanks Woj Kozicki!)
    # 2. Introduced configurable variable $N specifies flow-directory nesting levels
    # 3. Reduced default value of variable $flow_capture_interval to 1800
    # 4. Created FlowViewer_NDBM.cgi for users whose Perl does not have GDBM
    # 5. Created configurable 'work_directory' separate from cgi_bin_directory
    # 6. Sped up concatenation for requests that cross day boundaries
    # 7. Added filter fields: Protocol, TOS Field, TCP Flags
    # 8. Added some more syntax checking
    # 9. Added FlowGrapher capability (requires GD for Perl)
    #
    # Version 1.0 (Original)
    #
    #
    # Vital Assistance
    #
    # Special thanks to those FlowViewer users who provided feedback and valuable
    # suggestions, including Sejin Ahn, Mark Boolootian, Bogdan Ghita, Woj Kozicki,
    # Ed Arvin, Alex Shepherd, Mike Smith, Scott Wingfield, Vali Magdalinoiu, Ed
    # ravin, Eric Lautenschlaeger, Sean Cardus, Carles Kishimoto, Shigeki Taniguchi,
    # Dave Faught, Peter Hofwell, Dario La Guardia, Mike Stowe, Chris Spitzlay and
    # Mike Donnelly. Big thanks to fellow toiler in the NASA vineyard Mark Foster
    # for some detailed testing, excellent suggestions, and code to go along with it.
    # Thanks from all of us to NASA whose unending support of innovation in all fields
    # has resulted in this toolset.
    #
    # Bugs, recommendations
    #
    # If you need help installing, have a question, discover a bug, or have a
    # recommendation, please send an email to:
    #
    # Joe Loiacono
    # jloiacon@csc.com
    #
    # FlowViewer is being developed at NASA by a contractor in the employ of the
    # United States Federal Government in the course of his official duties.
    # Pursuant to Title 17, Section 105 of the United States Code, this software is
    # not subject to copyright protection and is in the public domain. FlowViewer is
    # an experimental system. NASA assumes no responsibility whatsoever for its use
    # by other parties, and makes no guarantees, expressed or implied, about its
    # quality, reliability, or any other characteristic.
    
    

    About Version 3.4
     Version 3.4 Release Notes
    
     It's been awhile, so version 3.4 will fix a myriad of little problems which
     I mostly can't remember. The primary new capabilities include:
    
      1. In most cases, the user may now switch the device without losing entered filter criteria
      2. The different tool logos now provide a link to the Saved Reports page
      3. Users can now provide a meaningful name for saved FlowViewer and FlowGrapher reports
      4. Fixes to an end-of-year problem have resulted in a 8% speed up of FlowGrapher in general
      5. Users can select to limit FlowGrapher stats to no-zero data points, if desired
      6. Fixed problems with sorting
      7. Corrected the graphing by 'flows' (was graphing 'flags' :-)
      8. Can now provide up to 20 source or destination IP address/address ranges
      9. Can now exclude specified IP addresses from a larger included address range
    
      Modifications have been made to FlowGrapher_Main.cgi since the original 3.4 distribution
      to fix a problem caused by the new speed-up processing. The speed-up was not accounting
      for Daylight Savings considerations.
    
     New Scripts and Files:
    
      FlowGrapherM.png            New logo link points to Saved reports web page
      FlowGrapherS.png            Revised logo link permits naming of Saved Reports
      FlowViewerM.png             New logo link points to Saved reports web page
      FlowViewerS.png             Revised logo link permits naming of Saved Reports
      FlowTrackerM.png            New logo link points to Saved reports web page
      flowcapture_restart         Renamed flowcap script for restarting flow-captures
      flowtracker_restart         New script for re-starting FlowTracker_Collector
    
     General Notes:
    
      Remember to copy into the new directory (e.g., /usr/lib/cgi-bin/FlowViewer_3.4) user
      logos, names file, as_names, NamedInterfaces_Devices, NamedInterface_Exporters,
      FlowViewer_SavedFilters, etc., from the old cgi-bin directory.
    
      The simplest way to transition to the new version is to leave all 
      FlowViewer_Configuration.pm settings alone except:
    
       $reports_directory       = "/var/www/FlowViewer_3.4"; 
       $reports_short           = "/FlowViewer_3.4"; 
       $graphs_directory        = "/var/www/FlowGrapher_3.4"; 
       $graphs_short            = "/FlowGrapher_3.4"; 
       $tracker_directory       = "/var/www/FlowTracker_3.4"; 
       $tracker_short           = "/FlowTracker_3.4"; 
       $old_tracker_directory   = "/var/www/FlowTracker_3.3.1"; 
       $cgi_bin_directory       = "/usr/lib/cgi-bin/FlowViewer_3.4"; 
       $cgi_bin_short           = "/cgi-bin/FlowViewer_3.4"; 
       $work_directory          = "/usr/lib/cgi-bin/FlowViewer_3.4/Flow_Working"; 
       $work_short              = "/cgi-bin/FlowViewer_3.4/Flow_Working";
       $names_directory         = "/usr/lib/cgi-bin/FlowViewer_3.4"; 
       $log_directory           = "/usr/lib/cgi-bin/FlowViewer_3.4"
    
      The following can remain the same (or else copy the contents to the new directory):
    
       $save_directory          = "/var/www/FlowViewer_Saves"; 
       $save_short              = "/FlowViewer_Saves"; 
       $filter_directory        = "/usr/lib/cgi-bin/FlowTracker_Files/FlowTracker_Filters";   
       $rrdtool_directory       = "/usr/lib/cgi-bin/FlowTracker_Files/FlowTracker_RRDtool";
    
      If this is an upgrade for you (e.g., from v3.3.1) I recommend using the FlowViewer_Relay.cgi,
      FlowGrapher_Relay.cgi, and the FlowTracker_Relay.cgi scripts to alert users to the new
      version with links and a reminder to change their bookmarks.  In each of the Relay scripts
      tailor the following line to your environment (i.e., point to the new 
      FlowViewer_Configuration.pm file):
    
       require "/usr/lib/cgi-bin/FlowViewer_3.4/FlowViewer_Configuration.pm";
    
      ... then, copy the following new Relay files overtop of the related files in your
      old cgi-bin directory (e.g., FlowViewer_3.3.1):
    
       cp .../FlowViewer_3.4/FlowViewer_Relay.cgi .../FlowViewer_3.3.1/FlowViewer.cgi
       cp .../FlowViewer_3.4/FlowGrapher_Relay.cgi .../FlowViewer_3.3.1/FlowGrapher.cgi
       cp .../FlowViewer_3.4/FlowTracker_Relay.cgi .../FlowViewer_3.3.1/FlowTracker.cgi
      
      Now, when users go to their book-marked FlowViewer web pages, they will be directed
      to the new ones. FlowTracker_Relay.cgi is particularly important if this is an upgrade;
      it copies over archived FlowTrackings which would be a bit tedious to copy by hand.
    
      The rsync_flows and rsync_trackings scripts are useful for easily backing up all
      raw netflow data and FlowTracker state information (Filters and RRDtool
      databases.) The FlowViewer_CleanFiles script is useful for deleting aging files
      that are not necessary anymore. I run it out of 'cron' once a day.
    
      The performance_check script can be used from the command line to keep track 
      of how well your implementation is performing. I run it against my
      Flowtracker_Collector.log file to see how things are going. Here at the
      NASA Earth Observing System network I have over 200 FlowTrackings and
      they complete in an average of 44 seconds. FlowTracker_Collector runs every
      five minutes and I watch for runs that take longer than five minutes.
      Even in those situations, however, FlowTracker_Collector seems to continue
      on with no real visible effects.
    
    About Version 3.3

    Version 3.3.1 fixes some problems related to using the new Exporter capability. A FlowGrapher problem with sorting host names has been fixed. A problem with FlowGrapher graphing across the midnight boundary is fixed. The FlowViewer_Relay.cgi optional scripts have been made easier to set up. A problem with FlowTracker and the use of $no_devices_or_exporters has been fixed. FlowViewer_Cleanup scripts have been cleaned up. A number of fixes have been added: exporter problems, FlowTracker at month boundaries, very long FlowTracker_Collector and ...Grapher runs freezing on Debian platforms (Debian Perl handles 'sleep' differently than Red Hat.)

    Version 3.3 introduces a number of new useful features including:

    1. Some devices will now have 'named interfaces' (thanks C. Kishimoto)
    2. The user can now save filters of interest and recall them later
    3. Data can now be analyzed by Exporter ID (in addition to device name)
    4. Users can now set thresholds on FlowTrackings, and be alerted
    5. Users can now sort FlowGrapher output based on column type
    6. FlowViewer now provides Pie Charts
    7. Capability added to apply a Sampling Multiplier to output
    8. FlowTrackings now have a '3 year' graph
    9. The user can now generate text listings of FlowTracker output
    10. Filtering on next-hop has been added
    11. Logging has been made more flexible (e.g., less data)
    12. Preserve latest three notations (was keeping first three)
    13. Can now specify and display time-zones
    14. A hook has been provided for a User Logo with link out of FlowViewer
    15. New file cleanup scripts have been added
    16. Unit Conversion capability has been added (thanks C. Kishimoto)
    17. Can now graph Flows, Packets as well as Octets (thanks E. Lautenschlaeger)
    18. Improved AS name resolution (thanks S. Cardus)
    19. New saved_directory for storing saved Reports and Graphs.

    Version 3.2 introduces FlowTracker groups. These groups merge copies of multiple previously defined individual trackings onto one graph. Improvements have been made to the speed of FlowTracker_Collector. All FlowViewer, FlowGrapher, and FlowTracker logos now have embedded links for quicker navigation between the tools. A new Autonomous System (AS) resolving capability has been added. A couple of fixes since the original v3.2 release:

    1. 04/02/07 - Fixed FlowGrapher vs FlowTracker 'Average' differences (Thanks Dario La Guardia)
    2. 04/11/07 - Fixed long DNS names problem

    Version 3.1 includes statistcal information (MAX, MIN, 95th PCT, AVG) with FlowGraphs, introduces the ability to archive and restart trackings, now permits queries longer than 30 days, permits a range of port numbers (e.g. ports 1024:1048) on queries, and fixed a number of small issues.

    Version 3.0 introduces FlowTracker, a tool that permits users to track flow_data subsets, defined by specified filter criteria, over long time periods This version also modifies the interaction of the scripts so that the create_FlowViewer _webpage and create_FlowGrapher_webpage scripts are eliminated. Users will now point their browsers to FlowViewer.cgi, FlowGgrapher.cgi, and FlowTracker.cgi scripts instead. These scripts will pre-load the data input pages with the current time.

    About Version 2

    Version 2.3 is a major rework of FlowGrapher to increase it's processing speed. A 10-fold increase in speed was accomplished by eliminating calls to Time::Local:timelocal for each flow record processed. Computations of flow_length and determining of appropriate buckets are now in-line as much as possible.

    Version 2.2 introduces the flow_select option on the input page. This new parameter allows the user to specify which flows should be included in the specified time period. It may be the case that large, long flows may not end prior to the specified end time. These flows would not be counted previously as a flow was included only if it's end-time was within the time period. The flow_select option now permits the user to specify the type of flows he would like to see (e.g., "Any part of flow is within time period", "Start-time of flow within time period", etc.) This is more important over shorter time periods. The default value is "Any part of flow is within time period".

    Version 2.1 fixes a concatenation processing time "improvement" which turned out to be necessary after all, and end-of-year processing for FlowGrapher.

    Version 2 of FlowViewer introduces FlowGrapher, has halved the prosessing time, and has introduced several new features. The changes include:

    1. Used pipe (|) instead of re-reading intermediate files (thanks Woj Kozicki!)
    2. Introduced configurable variable $N to specify flow-directory nesting levels
    3. Reduced default value of configurable variable $flow_capture_interval to 1800
    4. Created FlowViewer_NDBM.cgi for users whose Perl does not have GDBM
    5. Created configurable 'work_directory' separate from cgi_bin_directory
    6. Sped up concatenation for requests that cross day boundaries
    7. Added additional filter fields: Protocol, TOS Field, TCP Flags
    8. Added additional syntax checking
    9. Added FlowGrapher capability (requires GD for Perl)