How to bulk download images, videos, documents and other media from forum threads

WFDownloader App has a forum media downloader which is among its manual link search options. This tutorial will show you how to bulk download various media such as images, videos, audio, documents, and so on from forum threads. You configure it to download what you want and click the search button to begin finding the links. Downloading from password-protected sites is possible by using either the "User Login" button or importing cookies from your browser so that WFDownloader App can use the same session like it. Note that this tutorial is NOT about downloading forum threads for offline reading. It's just about downloading forum files/media from threads like images, attachments, etc.

Explaining the user interface

The image above is the fetch options pane. Its elements are described below:

• Fetch: This option helps you to select the types of files you want to download. You select the checkboxes corresponding to what you want to download. For example, if I want to download both images and videos, I will select both options. It's possible that what you want to download does not fall under the given defined categories (images, videos, audio, documents). In that case, do not select any option, which will allow all links to be returned, which can also be filtered.
• Link category: This option will allow you to filter the links fetched based on whether the links must reside on the forum site, reside on an external site, or both (no restriction).
• Custom filter: This will allow you to filter the links found based on keywords or patterns. That is, links (URLs) will be accepted or rejected based on whether they contain a certain keyword or pattern. Each keyword to be matched must be put on a new line in the textbox. For example, if in a forum thread about animals I only want to download links containing the word "dog" and "cat", I will select the option "Only links matching" and put dog and cat on each line. Note that for more complicated matches, WFDownloader App allows you to use regular expressions by allowing you to toggle the "R*" button.
• Automatically extract links from known external hosts: Many forums put links to external image hosts like Imgur, Imagebam, Flickr, etc. When this option is enabled, the forum downloader will automatically crawl those hosts and extract the full images from them.
• Skip thumbnail images: By default, the application will download all images. When this option is selected, thumbnail images will automatically be removed from the return links. Note that it does this by best effort.
• Thorough page scan: You shouldn't have to use this. You should only use it when the link search isn't returning some expected links on the thread.
• File size filter: Use this to filter the result based on the sizes of the files.
• User Login: Use this button for forums where you need to log in to access the files. An alternative to this is to import cookies from your browser.
• Advanced Settings: This is for advanced control over the link search.

In the image above to the far right, you will notice 4 clickable buttons: save, load, export, and import buttons.

• Save: Use this to save a crawl configuration so that you can reuse it next time. In the process, you can optionally attach it to a website or URL pattern, so that next time you don't have to come here but instead click the blue "Add" button and search straight away! It will automatically reuse your saved configuration. This is useful for forum sites where you frequently need to download from their threads.
  
• Load: Use this to load a crawl configuration that you have previously saved.
• Export: Use this to export one or more forum crawl configurations so that you can share with others. You select the ones you want and then click "Export".
• Import: Use this to import forum crawl configurations from a file. Select the file and then click "Import".

The result section (which consists of the filter area) is below the above-mentioned section and works the same way as described in the manual link search tutorial linked earlier.

An example

In this example, I will be downloading pictures of cars from a forum thread with linked images from external hosts.

1. Go to the forum thread you want to download from. In this case, I will be using https://www.supercars.net/forum/threads/cars-in-paris.168820/

   

2. Next set the options you want. If you need to log in via application, you can do that now. I aim to download only the externally linked Flickr images without thumbnails. See my selected options below:
   
3. Click the search button to begin the link search.
4. Once the search is completed, you can do some post-search modifications like filtering results or generating custom names which I did.
   
5. After doing your modifications (if any), click "Confirm". If you have disabled/unchecked the "skip pre-batch creation step" from general settings, you will be able to set the initial batch properties as I have done below. I had to do this so that all files will end up in the same root folder (which is not the default).
   
6. A new batch will be created for you. Click the "Start" button to begin downloading the batch.
   
7. All that's left is for the batch to complete. You can right-click the batch box icon or any of its download items and select "Open Folder" to open the batch folder in your file explorer.
   
8. That's it.

Some crawling tips

1. Try not to violate the rules for crawling the forum you want. For example, it's always advisable to set reasonable crawl delays which can be done from the general settings. This way you won't burden the site you are crawling otherwise some forum sites may ban you (sometimes permanently).
2. Save the crawl settings you use often so that the next time, all you have to do is to load them.
3. You should leave the "Fast mode" option selected. If selected, the crawler will use the file extension to determine the file type. Unselecting it will drastically slow down the search because every candidate result will be connected to. You should only unselect the option when the website you are trying to download from doesn't make the file type obvious from the link. For example, a site can have an image link http://abc.com/image.png while another might have an image link of http://xyz.com/file. In the second one, it isn't obvious that the file is an image, and is a good case to unselect the option.
4. For the custom filter box, prefer the "Only links matching" filter option to the "Exclude links matching" option as the latter option is less restrictive and you may end up getting results you didn't expect.

Conclusion

We have seen that it is possible to bulk download forum media using WFDownloader App. All you have to do is configure it to download what you want. If you're having issues with this tutorial or have any suggestions, you can contact WFDownloader. Thanks for reading!

You may also like:

• Using manual search option to download from unsupported sites
• Using crawler mode for advanced link search