Web scraping with php
Web scraping is the process of extracting data from websites. It is typically done using a computer program that automates the process of visiting websites, parsing the HTML code, and extracting the desired data. Web scraping can be used for a variety of purposes, such as collecting product prices, tracking competitor activity, or creating market research reports. There are a variety of tools and techniques that can be used for web scraping. Some popular tools include Python libraries such as BeautifulSoup and Scrapy, as well as online services such as ScraperAPI and Octoparse. The specific tool or technique that is best for a particular task will depend on the complexity of the data that needs to be extracted and the frequency with which it needs to be updated. It is important to use web scraping responsibly and ethically. This means following the terms of service of the websites that are being scraped, not overloading the websites with requests, and giving credit to the original source of the data. Web scraping can be a powerful tool, but it is important to use it in a way that does not harm the websites that are being scraped.
Recommended
More Related Content
Similar to Web scraping with php
Similar to Web scraping with php (20)
More from Chakrit Phain
More from Chakrit Phain (20)
Recently uploaded
Recently uploaded (20)
Web scraping with php
- 1. Web data extraction with PHP Chakrit Phain Softnix Technology Co.,Ltd.
- 3. Topic HTTP Programming • Methods • Cookie • Session HTTP Tools • Chrome Develop tool • Postman HTTP access with PHP • Fopen • File_get_contents • Curl HTML access with DOM PHP I/O Stream
- 4. • Data extraction is the act or process of retrieving data out of (usually unstructured or poorly structured) data sources for further data processing or data storage (data migration). https://en.wikipedia.org/wiki/Data_extraction
- 5. Techniques • HTTP programming • DOM parsing • Text pattern matching (Regular expression) • Etc. https://en.wikipedia.org/wiki/Web_scraping
- 6. HTTP Programming • Methods • Get • Post • Cookie • Session
- 7. HTTP Request & Response GET /index.html HTTP/1.1 Host: www.example.com HTTP/1.1 200 OK Date: Mon, 23 May 2005 22:38:34 GMT Content-Type: text/html; charset=UTF-8 Content-Encoding: UTF-8 Content-Length: 138 Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT Server: Apache/1.3.3.7 (Unix) (Red- Hat/Linux) ETag: "3f80f-1b6-3e1cb03b" Accept-Ranges: bytes Connection: close <html> <head> <title>An Example Page</title> </head> <body> Hello World, this is a very simple HTML document. </body> </html>
- 8. Hand On #1 http telnet
- 9. Hand On #1 GET / HTTP/1.1 Host: testing-ground.scraping.pro HTTP/1.1 200 OK Date: Fri, 18 May 2018 05:17:36 GMT Server: Apache/2.2.22 (Debian) X-Powered-By: PHP/5.4.4-14+deb7u12 Vary: Accept-Encoding Content-Length: 3701 Content-Type: text/html
- 11. Cookie & Session http://www.hackingarticles.in/beginner-guide-understand-cookies-session-management/
- 12. Tools
- 13. Hand On #2 http cookie
- 15. How to get html content with PHP •fopen() •file_get_contents() •curl
- 16. Fopen
- 17. Hand On #3 Fopen
- 19. Hand On #4 File_get_contents
- 20. CURL
- 21. Hand On #5 CURL
- 23. HTML DOM Access • getElementById • getElementByTagName • DOMXPath
- 26. PHP DOM CLASS TYPE • DOMDocument • Represents an entire HTML or XML document • DOMNodeList • The NodeList object represents an ordered list of nodes • DOMNode • Each of DOMNodeList • DOMElement • Extend DOMNode
- 27. DOMXPath Expression Description nodename Selects all nodes with the name "nodename" / Selects from the root node // Selects nodes in the document from the current node that match the selection no matter where they are . Selects the current node .. Selects the parent of the current node @ Selects attributes
- 28. DOMXPath Path Expression Description bookstore Selects all nodes with the name "bookstore" /bookstore Selects from the root node bookstore/book Selects all book elements that are children of bookstore //book Selects all book elements no matter where they are in the document bookstore//book Selects all book elements that are descendant of the bookstore element, no matter where they are under the bookstore element //@lang Selects all attributes that are named lang /bookstore/book[1] Selects the first book element that is the child of the bookstore element.
- 29. DOM Example
- 31. Hand On #6 IMDB
- 32. PHP I/O Stream
Editor's Notes
- 3-5m
- สกัดเป็น csv
- Review Develop tools & Postman (5Mins)