Batch link search - How to queue multiple link extractions at once

This tutorial will show you how to queue multiple link searches/extractions at once using WFDownloader App. This is the batch link search feature and can be found under the "Tasks" menu. While WFDownloader App can help you extract and download links from almost any website page, there are times when you want to do it multiple times. For example, you give it the link to an artist's profile on Twitter or ArtStation and it extracts and downloads the images. You have another one, you repeat the same process. What if you have several artists (10, maybe 100) whose images you want to download? Surely doing it one by one will become tedious. WFDownloader App allows you to import a list of links you want it to search and then download from them as a batch. You can also use it even if you only want to extract the links without downloading them. In the given scenario above, just compile a list of artists' profiles you want into a list and then feed that list into WFDownloader App. It will go through each one, perform the needed link search or extraction, and then download the links found.

Batch link search example

Follow the steps below:

1. Compile the list of pages you want to search/extract/download links from.
2. Select all the links and copy them to your clipboard.
   
3. Open WFDownloader App, go to the "Tasks menu" and select the "Batch link search" option. Notice the links have automatically been pasted in for you.
   
4. If it doesn't, you can just copy and paste the links into the text box provided or click "Grab from file" to import the links from a file.
   
5. Enter the batch name and the folder to save the downloaded links.
   
6. If you need to change certain options such as adding delays to the link search/download, setting how many links to download simultaneously, download conflict actions, etc., the click "More options" button for that and click "Accept" to save the changes.
7. Click "Confirm". A new window will be opened with the link search started automatically. Each link will be processed and the corresponding links found and downloaded.
   
8. Once all links have been processed, you will get a notification. You can check either the "Failed Downloads" or "Search Summary" tab to see which downloads failed or which link searches/extractions failed and the reasons given.
   
9. If there are failed downloads or failed link extractions, you may click "Re-run" to retry the failed ones. In this example, two of the link searches failed due to "no links being found" (i.e., no media) as shown in the image below. I won't rerun since I already knew this beforehand and only included them to make this example more detailed.
   
10. Click "Accept" if you're satisfied with the operation. A batch will be created for you with all the links that were downloaded. That's it!
    
11. You can open the folder and see the files that were downloaded.
    

Batch link search options explained

• Batch mode: If you select the first option, all link searches are treated as a single operation and then combined to give you one batch at the end after you click "Accept". In the example above, you will notice only one batch was created at the end. If you select the second option, each of the links is treated as separate batch operations resulting in multiple batches created at the end. For example, if the second option was chosen in the above example, instead of only one batch being created, five would have been created at the end - one for each successful link search. To help you decide, use the first option if this is going to be a one-time operation, and choose the second if you would like to update each created batch later on (e.g. for artists that update frequently).
• Extraction mode: You can select whether you want the links that were found to be downloaded automatically or you can just extract the links only in case you are not interested in downloading the links but want to use them in another application.
• Site config: This will be enabled if you are trying to perform a batch link search operation on a site that has configurable options. If you look at the example above, you will notice it was enabled for Twitter since it has configurable options. An example of what it looks like is shown below.
  
• More options: This gives you more options to control the actual batch link search options. For example, if a site is blocking you (e.g. you notice only the first few searches completed while all the others failed), you may need to set delays either in the link search or in the link download, or both.
  

Conclusion

We have seen how to queue multiple link search operations via the batch link search mode. We've also been warned that not all sites like this and may block one's IP address or one's account (sometimes permanently) for the site. Sometimes, it's just a matter of setting appropriate delays so that the site doesn't block you. Other times, the site you want to batch link search from may forbid it, so check. If you notice that after the batch link search only some succeeded while others failed, it's not that the software is buggy. It is probably because you are being blocked so you may have to check whether the site forbids this.

If you have any issues or suggestions for improvements, you can either comment below or contact us. Thanks for reading!