Run from CLI
DeepTrawl can be run directly from the CLI on both Windows & Mac. This feature allows you to integrate it into your workflow using scripts.
Because DeepTrawl also supports automatic archiving of results and Profiles it's use can be completely automated.
The syntax is ultra simple, just use deeptrawl -trawlProfileName <yourProfileName>. Because the named profile contains all of DeepTrawl's settings, the CLI can be used to trawl trawl multiple sites, using all of their individual settings.
Get your free trial for Windows & Mac