It is unable to scrape <li>

[amztc34283]

wanted_list = ["Design, develop, test, refactor and scale backend implementations of new and existing consumer product features"]

scraper = AutoScraper()
result = scraper.build(url, wanted_list)

I am able to scrape the element in the wanted_list but similar elements are not scraped successfully, any tips and tricks could fix this?

[alirezamika]

please provide your full code including the url.

[amztc34283]

wanted_list = ["Design, develop, test, refactor and scale backend implementations of new and existing consumer product features"]
scraper = AutoScraper()
result = scraper.build(url, wanted_list)
print(result)

Link: https://careers.chime.com/en/jobs/4225356002/backend-engineer/

[alirezamika]

What is your expected output?

[amztc34283]

My expected output is the content of all the <li> under the same <ul> which is:
Design, develop ...
Work with ...
Collaborate with ...
Proactively find ...

[alirezamika]

you can try the contain_sibling_leaves attribute.

result = scraper.get_result_similar(url, contain_sibling_leaves=True)

[amztc34283]

I will give it a try, thanks.

In addition, can you point me to the part of the code that decides which elements to scrape based on the wanted_list? Thank you!

[alirezamika]

its basically the whole code 😅