All Questions
Tagged with web-crawler xpath
251 questions
0
votes
2
answers
84
views
Is there a reason why I can't extract the price data from the Nike Website using selenium and Python
I am currently building a Python script that takes the price of trainers from the Nike website and pushes the price into a CSV file. Originally, the code took the element where the price data lives ...
- 3
1
vote
2
answers
35
views
Xpath for when positioning is never fixed
I've been using a specific Xpath for scraping data using a crawler. The issue is that the Xpath is looking for an "li" tag in a specific oder/position.
When other "li" tags appear ...
- 13
0
votes
1
answer
21
views
How to search only child elements in xpath, not grandson elements
I only need to count the number of child tr, not the number of grandson tr, but my current count is 8, and the result I want to get is 2. I am a newer, how to solve this problem?
from lxml import ...
- 13
0
votes
1
answer
115
views
Typing on an Input element textbox with selenium python
Typing in an Input element textbox in Python Selenium is showing TimeoutException. Here is the code.
#provides access to the webdriver
from selenium import webdriver
#allows interraction with elements ...
- 13
1
vote
2
answers
59
views
xpath works in console but doesn't work in scrapy
I'm learning web scraping and I'm trying to scrape this website http://wiki.stat.ucla.edu/socr/index.php/SOCR_Data_MLB_HeightsWeights
so when I do "scrapy crawl baseket " the results is ...
-2
votes
1
answer
42
views
why xpath output keeps changing?
the problem i am facing is weird and its so much waste of time
in theory this should give out the link
next_page = response.xpath('//ul[@class="pagination justify-content-center"]/li[6]/a/@...
user21195292
0
votes
1
answer
36
views
Python selenium error: no such element: Unable to locate element; clicking a button on the screen
I've been trying to figure out this error for serveral hours until now..
I tried to click the red button on the screen, but somehow I can't use the xpath method.
The error message waas given right ...
- 1
-1
votes
1
answer
216
views
I am trying to scrape the image src using scrapy in python but instead, form img element want to scrape from <source> element that has no class
I am trying to scrape the image src using scrapy in python but instead, form img element want to scrape from element that has no class attribute or src attribute, can anyone please help me with how ...
- 135
0
votes
1
answer
248
views
XPATH: .select_by_visible_text not working
This seems like a really trivial problem, but for some reason my .select_by_visible_text isn't working on my form. Here is the XML layout:
<label for="urn:li:fs_easyApplyFormElement:(urn:li:...
- 23
0
votes
2
answers
89
views
Not able to find the Xpath
I am trying to scrape IMDB top 250 movies using scrapy and stuck in finding the xpath for duration[I need to extract "2","h","44" and "m"] of each movie. ...
2
votes
1
answer
1k
views
Puppeteer: select script tag containing certain text
I'm trying to crawl a facebook photo page which contains several script tags. I want to select a specific one which contains "nextMedia". I did this:
code1 = await page.$x('//script[contains(...
- 43
0
votes
2
answers
131
views
Scrapy crawls no pages
I am currently trying to crawl Rolex watches on chrono24 page with the spider called chrono24, but scrapy crawls 0 pages. I have discovered via the shell response.css('div.article-item-container.wt-...
- 57
0
votes
1
answer
61
views
VBA runtime error in using findelementsbyxpath
xpathstring
="/html/body/div[2]/section[2]/section/section/div/div[2]/div[1]/div[1]/div[1]/div[2]/div/div/table/tbody"
j = 1
trcnt = 1
For Each ele In cd....
0
votes
1
answer
105
views
Webscraping customer review - Invalid selector error using XPath
I am trying to extract userid, rating and review from the following site using selenium and it is showing "Invalid selector error". I think, the Xpath I have tried to define to get the ...
- 21
0
votes
1
answer
1k
views
How to crawl question and answer of Google People Also Ask with Selenium and Python?
I used this code for crawl question and anwser of Google People Also Ask. I want use that for create idea for writer.
But I can't get exactly that element, in this time I was try with full Xpath, but ...
