How to program WFDownloader App - Introduction

WFDownloader App is a powerful downloader that even has its own coding/scripting language. It's the first truly programmable downloader! Have you ever been in a situation where you requested for a new site to be supported in your favorite downloader tool and it fell on deaf ears? Alright, maybe not on deaf ears. Your request was acknowledged, but months after, you check again and the status is still... just acknowledgment! You understand that the maintainers of the tool might be busy or overwhelmed with requests and yours is just not their priority. You'd like to contribute, but Java or Python seems a little too much for you. If only they were a little easier? That's where the WFDownloader Scripting Language (WSL) comes into play. It is a Domain Specific Language (DSL) targeted at downloading so that it becomes very easy to do in the application. It's a very concise scripting language, yet very powerful. It has been tested on several hundreds of websites so it's able to provide solutions for common tasks. It can be used to download images, videos, audio files, hyperlinks, documents, or any kind of file. For example, you can program it to download only large images that have a particular pattern while filtering out tiny thumbnails. Going technical, it supports link filtering, URL rewrites, infinite scrolling, gallery download, multi-page crawls, captcha handling, custom naming, data scraping, and other features. Over 90 percent of the supported sites in WFDownloader App were programmed in the same way you are about to learn, so it's quite capable.

Why is this important?

You might be wondering why you should pay attention to this? The thing is, it's impossible for any download software (including WFDownloader App) to support every single website on the internet. Most tools support only popular sites and if your site isn't popular enough, it may never get supported and so you may never get an easy way to download from there. If it were easy, you'd just support it yourself but using a full-blown programming language like Java or Python can be intimidating for a non-programmer and might take a lot of time and effort. That's why WFDownloader App has greatly simplified the process using concepts that even non-programmers can understand. For example, if you are trying to grab some images off a page, I don't think I need to explain what the code "fetch first 5 images;" would do. Here are some of the benefits:

1. The code syntax is simple and concise for most people to understand thereby enabling them to support new sites within minutes.
2. It allows you grab just the files you want at once without any extra effort on your part e.g. having to delete thumbnails afterwards, or having to manually select the files you want from a list.
3. You only need to code it once and attach the code to a website or URL/link pattern. After that, you will be able to download seamlessly from other links of that website as if it were already a supported site. That's because the app will automatically reuse and apply your code.
4. Even if you are programmer, you could save yourself time and effort because of how easy it is to program and reuse.
5. Apart from being one of the ways to extend WFDownloader App to support new sites, you can export your code and share with your friends. Once they drag and drop your file onto their main window, they'll also be able to download from the new sites thanks to you!

Getting started

All you need to get started is to download WFDownloader App and install it. You don't need to install any additional tools or packages, tamper with your OS system variables, import libraries, and you certainly do not need to experience dependency hell!

Explaining the user interface

Link Address area

This is the top section of the user interface. It consists of the following controls:

• Link Address: Enter the URL/link of the page you want to use to test your code.
• Save Folder: Enter where the fetched links should be downloaded into. You can use the "Browser" button beside it to select a particular folder.
• Search: Click this button to begin a link search with the code you've written.

Code area

This is the section where you write the link fetch code. This section is collapsible so that you view more of the results table. It consists of the following:

• Code Editor: You write the code in this text area.
• Save: Use this to save the code you've written so that you can reuse it next time. In the process, you can optionally attach it to a website or URL pattern, so that next time, you don't have to come here but instead click the blue "Add" button and search straight away! It will automatically reuse your saved code.
• Load: Use this to load some code that you have previously saved.
• Export: Use this to export one or more saved link fetch codes you've written so that you can share with others. You select the ones you want and then click "Export".
• Import: Use this to import from a file. Select the file and then click "Import".
• Toolbar area: This consists of the following:
  • Run: Click this to run the code you've written. It's the same as clicking the "Search" button. If the code you've written is without grammatical error, you'd see a green indication at the top otherwise you'd see a red one indicating that an error has occurred. For now, the errors displayed aren't very user-friendly. If more users indicate interest in this feature,it will be improved.
  • Log: View the log of what transpired when you ran your code. It can help to debug issues you encounter.
  • Login: If the site needs a login, you can use this button to do so. However, it's preferred you import cookies instead as that will persist even after you close and restart the application.
  • View Html: Use this to view the HTML source code of the page provided. Sometimes you may need to look at the page's HTML source code why your code isn't picking up specific links.
  • Examples: Find example WSL codes you can play with. Note that the link address text field will have its link replaced with one that is compatible with the example code. The examples also have explanations in them. Click "Run" to see the code in action.
  • Quick Help (??): This will give a quick overview of the programmable mode.
  • Editor Expander ([...]): This can be used to expand the code view via toggling to help your viewing experience. Once expanded, the results table will be hidden until you toggle it back to the default.

Search Results section

The rest of the window is the search results section which consists of the table where the links are populated after each code run. This section is common to all manual link search options and has already been explained here.

The general workflow of programming WFDownloader App

If you've already installed WFDownloader App, the general workflow is like this:

1. Go to the "Tasks" menu and click "Add batch download via manual links fetcher".
   
2. Provide a sample link address (of the site which you want to code against), enter a save folder, select "Programmable mode" and click "Next".
   
3. Enter the code that grabs the sorts of links you want.
   For this illustration, click the "Examples" dropdown button and select "Image fetch with filter". Now replace the word "jackal" with "media" as shown below. Notice there is a semi-colon ";" at the end so make sure it's there or you'll get an error.
   
   
4. Click "Search" or "Run" to test your code.
   
5. If there's an error in the code you've written, you will get an error message in red. Use that to fix the code. Otherwise, you'll get an okay message in green.
   
6. Check the links found and confirm whether they are what you expect.
   In this illustration, all the image links have the word media in them, which is what we want.
   
   
7. Try it out on multiple sample links (from the same site) and tweak your code until you are satisfied with the results.
   In this tutorial, we'll skip this step.
8. Now, save the code and attach it to a website or URL pattern.
   The URL pattern must be a regular expression. In the image below, "Generate it for me" was selected.
   
9. After this, you're done, and will no longer need to use this section for the site or for the URL pattern that you entered. Close the window and head back to the main window of WFDownloader App.
10. Click the big blue "Add" button, enter a conforming site link to what you've just coded and search and it will automatically use the code you've saved.
    
    

If you were able to complete the above steps, you've successfully supported a new site all by yourself. You can export newly supported sites so that you can share with your friends that may not be as tech-savvy. All they will have to do is drag and drop your exported file onto the main WFDownloader App window and they will also be to download from the new sites thanks to you!

A few sample codes (including real-world examples)

Find in the image below a few sample codes that illustrate programming in WFDownloader App. Notice how simple and succinct the syntax is. Even without explaining anything, some of you can already understand what is written.

To see examples of WFDownloader Script used in real websites such as bing, bunkr, cyberdrop, imagetwist, 4chan, etc., see this GitHub repository.

Conclusion

The programmable mode feature of WFDownloader App is one way to support new sites in the application. In summary, you open the code editor, write the code, save and attach it to a website, and then you can start downloading from the site seamlessly. This was just an introduction. There wasn't really any explanation of what the code was doing. It was just about walking you through the process. We'll go into detail in the next tutorial, which will be coming soon. For now, you can explore the other examples in the application. If you like, you can watch a video version of this tutorial below. If you have any question, suggestion, or feedback, the comment section is all yours. Thanks for reading!