Thank you so much for this add-in, it’s really useful! I’m having trouble with a task, and I’m not sure if it’s possible. Can I scrape a profile page and then have the recipe click back to the main person list and then click and scrape the next profile?
Thank you!
Hi John,
Thanks for joining the community! Scraping data that you have to click into to see is very doable. But rather than clicking back and forth, Data Miner actually uses URLs for its’ automation. This process is called a Crawl. The process is outlined in this video: Crawl Scrape 10/9/2020 - YouTube and we have further documentation on our help page: Crawl and Scrape webpages | Data Miner.
This process requires two recipes. You need to scrape the detail page URLs off of the search results page using one recipe and then Data Miner visits the URL and then scrapes the detail page’s data using the second recipe. You can make the recipes yourself by looking at our tutorials here: Writing Recipes with Recipe Creator | Data Miner
And if you need any help with writing recipes please post your questions to the “Recipe Writing Help” category here on our form.
Please let me know if you have any questions.
Thank you, Zach. I will take a look at these resources and let you know if I have any questions.
Thanks for all your help today Zach!