Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Or you could’ve used xpath and bs4 and have been done in an hour or two and have more understandable code.


it is not that easy, there is lazy loading in the page that is triggered by scroll of specific sections. You need to find clever way, no way to scrape with bs4, so tough with even selenium.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: