Overview
Previously we featured the 404Project https://isc.sans.edu/diary/ISC+Feature+of+the+Week+The+404Project/12415 As we mentioned, the main purpose of this project is to trend the web pages crawlers and automated bots are trying to access.
We've had a good number of submitters add this script to their error page and have been collecting data for a while now. We made a few summary reports to get started trending the information. The project has been moved to its own space but the old pages should still get you to https://isc.sans.edu/404project/.
Report Information
The summary reports can be viewed at https://isc.sans.edu/404project/reports.html. The page is generated once a day for the previous day's data. The tables have descriptions for each field so I will just list them and summarize here.
Daily Totals
Complete summary totals for the given date
Top 10 Submitted URLs
Note the percentage is based on the max a particular URL has been submitted meaning if the submissions that particular day have been the most ever, the graphic will be out at 100%
Top 10 User Agents Submitting
User Agents with counts and unique submitters and URLs
API Information
We've also added a couple API interfaces if you'd like to view previous data.
https://isc.sans.edu/api/#daily404summary
Each days totals. Accepts date and limit.
https://isc.sans.edu/api/#daily404detail
Each days details. Accepts date and limit. (Look for more fields output in the future)
Let us know in the section below if you have suggestion or feedback about these preliminary reports or send us any questions or comments in the contact form at https://isc.sans.edu/contact.html
--
Adam Swanger, Web Developer (GWEB)
Internet Storm Center (http://isc.sans.edu) (c) SANS Internet Storm Center. http://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.
Previously we featured the 404Project https://isc.sans.edu/diary/ISC+Feature+of+the+Week+The+404Project/12415 As we mentioned, the main purpose of this project is to trend the web pages crawlers and automated bots are trying to access.
We've had a good number of submitters add this script to their error page and have been collecting data for a while now. We made a few summary reports to get started trending the information. The project has been moved to its own space but the old pages should still get you to https://isc.sans.edu/404project/.
Report Information
The summary reports can be viewed at https://isc.sans.edu/404project/reports.html. The page is generated once a day for the previous day's data. The tables have descriptions for each field so I will just list them and summarize here.
Daily Totals
Complete summary totals for the given date
Top 10 Submitted URLs
Note the percentage is based on the max a particular URL has been submitted meaning if the submissions that particular day have been the most ever, the graphic will be out at 100%
Top 10 User Agents Submitting
User Agents with counts and unique submitters and URLs
API Information
We've also added a couple API interfaces if you'd like to view previous data.
https://isc.sans.edu/api/#daily404summary
Each days totals. Accepts date and limit.
https://isc.sans.edu/api/#daily404detail
Each days details. Accepts date and limit. (Look for more fields output in the future)
Let us know in the section below if you have suggestion or feedback about these preliminary reports or send us any questions or comments in the contact form at https://isc.sans.edu/contact.html
--
Adam Swanger, Web Developer (GWEB)
Internet Storm Center (http://isc.sans.edu) (c) SANS Internet Storm Center. http://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.