Grabbing urls on a directory hidden by javascript

How do I scrape...?
#1

Little bit out of my wheelhouse because I have not run into this situation before where the tag is not in the code and neither is the url that each box on this page has, it appears to be generated once you click on one of them.

This is an example of a directory I was trying to scrape: https://ensun.io/search?q=RFID&locations=United%20States%2Cnull%2Cnull

When you click on one of the boxes, it goes to their specific profile page. I was hoping to scrape/crawl this website. I just don’t know how to achieve that.

Any ideas?

#2

Hi There,

Thanks for posting! I reviewed the site and this page is challenging, but crawling it should still be possible. You will have to build your URLs from information that is available on the page. The detail page URL consists of a unique ID and the company name.

It appears the unique ID for each company is found in the logo URL. So you will have to extract this logo URL and also the company name. Then combine them to make the detail page URL. This can be done later in excel or within Recipe Creator.

At this level of recipe development, I might recommend a custom recipe created by our team. Custom Recipes cost $150 per recipe and take 2-3 days to write. You can learn more about them here: Custom Recipe Development Service | Data Miner

Please let me know if you have any questions.

#3

I see now, the logo image url has the unique id at the end and I would need to pull the company name and that url in order to form the page urls I need to crawl. Clever!