Minding the Web - iMorph's InfoMinder automates website change-tracking by John Udell
John Udell

InfoMinder is a service that consolidates newsletters, syndicated newsfeeds, and manual website tracking into a single easy-to-use solution.

A recent article in eWeek¹ focused on network security. The article include a helpful sidebar, eWeek Labs Recommends: Key security resources, which listed fourteen URLs and suggested: "Every IT pro should bookmark and visit these sites -- often." Of course they should. Likewise, workers in every area of the information economy should keep a close eye on the websites vital to those areas. The question is: "How?" More specifically:

• How can I find the time to do this monitoring?

• How can I consolidate all this monitoring into a single activity?

• How can I can I focus on all the changes, and only the changes?

The web offers a variety of monitoring solutions, and in a way that's part of the problem. Some sites enable you to register for email alerts, but each signup procedure is different, and dozens of alerts from dozens of sites can swamp your email inbox. Other sites syndicate newsfeeds to aggregators, and when that's true, you can save a lot of time and consolidate your monitoring activities. But news syndication, though practiced by some sites (notably, technology sites), is not yet a reliable standard. So if you want to stay current in an area, and really cover the area exhaustively, you're stuck with a cumbersome mix of strategies. You sign up for email alerts when available, you track newsfeeds when available, and the rest of the time you visit sites one at a time. Then, it's up to you to synthesize all these information flows in order to build up a complete picture.

The solution: iMorph's InfoMinder

InfoMinder is a service that tracks changes on a set of web pages that you specify. If you were tracking the fourteen security-related websites mentioned in that eWeek article, an email notification from InfoMinder would look like the following example.

Figure 1: Sample InfoMinder notification email

InfoMinder's notification email summarizes changes on all tracked pages. The View All Changes link leads to a special page that highlights all changes on each tracked page.

Change tracking: sometimes, there's more than meets the eye

Clicking the View All Changes link leads to a version of the tracked page on which just the changes are highlighted. The following example shows the two changes reported for the Microsoft TechNet Security Bulletin. One of the changes appears where you'd expect to find a recent change (in the June listing), as seen in Figure 2.

But the other change is buried further down the page (in the January listing), as seen in Figure 3.

Figure 3: Monitoring the Microsoft TechNet Security Bulletin, continued

The MS01-002 vulnerability, though listed under January
changes, did not in fact appear on the page until June.

Why did this happen? Probably, the MS01-002 item was forgotten in January, then added in June. The point is that although websites often order items chronologically, you can't really know what has changed without the kind of before-and-after comparisons that InfoMinder makes.

Syndicated news: InfoMinder can enhance and complement RSS channels

When available, syndicated newsfeeds are an excellent way to track website changes. Some of the sites on eWeek's list, including SecurityFocus.com, do offer newsfeeds that can be monitored in aggregators such as Meerkat.

Some sites, notably technology-oriented ones, export newsfeeds that can be searched and viewed in aggregators. Here, the viewer is Meerkat.

In this case, InfoMinder's email summary reported the same changes reported by Meerkat. And, as shown in Figure 5, InfoMinder's View All Changes page managed to add value above and beyond what Meerkat reported, by including a summary of the Hacker Tools and their Signatures story. The summary appeared on the website, but was omitted from the newsfeed.

Figure 5: InfoMinder can provide a richer view than that of a syndicated news feed

InfoMinder's change view includes a helpful summary which was omitted from the RSS newsfeed.

It's interesting to see how InfoMinder automatically gathers and presents the summary which could have been included in the newsfeed's <description> field, but wasn't.

The broader point, in any case, is that only some of the fourteen sites mentioned in the eWeek story are available as syndicated channels. InfoMinder consolidates the tracking of all the sites, whether or not they support syndication.