Update module github.com/gocolly/colly to v2 #47

Open
renovate-bot wants to merge 1 commit from renovate/github.com-gocolly-colly-2.x into master
Collaborator

This PR contains the following updates:

Package Type Update Change
github.com/gocolly/colly require major v1.2.0 → v2.3.0

Release Notes

gocolly/colly (github.com/gocolly/colly)

v2.3.0

Compare Source

v2.2.0

Compare Source

Tons of bug fixes & little improvements & updated dependencies

What's Changed

New Contributors

Full Changelog: https://github.com/gocolly/colly/compare/v2.1.0...v2.2.0

v2.1.0

Compare Source

  • HTTP tracing support
  • New callback: OnResponseHeader
  • Queue fixes
  • New collector option: Collector.CheckHead
  • Proxy fixes
  • Fixed POST revisit checking
  • Updated dependencies

v2.0.1

Compare Source

v2.0.0

Compare Source

  • Breaking change: Change Collector.RedirectHandler member to Collector.SetRedirectHandler function
  • Go module support
  • Collector.HasVisited method added to be able to check if an url has been visited
  • Collector.SetClient method introduced
  • HTMLElement.ChildTexts method added
  • New user agents
  • Multiple bugfixes

Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate CLI.

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/gocolly/colly](https://github.com/gocolly/colly) | require | major | `v1.2.0` → `v2.3.0` | --- ### Release Notes <details> <summary>gocolly/colly (github.com/gocolly/colly)</summary> ### [`v2.3.0`](https://github.com/gocolly/colly/compare/v2.2.0...v2.3.0) [Compare Source](https://github.com/gocolly/colly/compare/v2.2.0...v2.3.0) ### [`v2.2.0`](https://github.com/gocolly/colly/releases/tag/v2.2.0) [Compare Source](https://github.com/gocolly/colly/compare/v2.1.0...v2.2.0) Tons of bug fixes & little improvements & updated dependencies #### What's Changed - Added new project that uses Colly by [@&#8203;twiny](https://github.com/twiny) in [#&#8203;525](https://github.com/gocolly/colly/pull/525) - adding docker-slim to the list of open source projects using Colly by [@&#8203;kcq](https://github.com/kcq) in [#&#8203;526](https://github.com/gocolly/colly/pull/526) - update coursera example by [@&#8203;ysung6](https://github.com/ysung6) in [#&#8203;537](https://github.com/gocolly/colly/pull/537) - Update openedx_courses.go by [@&#8203;lingt-xyz](https://github.com/lingt-xyz) in [#&#8203;539](https://github.com/gocolly/colly/pull/539) - feat: support Stop() in queue by [@&#8203;PalanQu](https://github.com/PalanQu) in [#&#8203;531](https://github.com/gocolly/colly/pull/531) - Fix invalid base URL assignment by [@&#8203;dasshield](https://github.com/dasshield) in [#&#8203;549](https://github.com/gocolly/colly/pull/549) - Add context.Context support by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;548](https://github.com/gocolly/colly/pull/548) - Fix handling of redirected URLs in OnResponseHeaders by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;561](https://github.com/gocolly/colly/pull/561) - Fix race on count fields of Collector type by [@&#8203;shubham14bajpai](https://github.com/shubham14bajpai) in [#&#8203;563](https://github.com/gocolly/colly/pull/563) - SetProxyFunc: fix proxies rotation by [@&#8203;wowinter13](https://github.com/wowinter13) in [#&#8203;567](https://github.com/gocolly/colly/pull/567) - Update xkcd_store.go by [@&#8203;BigManing](https://github.com/BigManing) in [#&#8203;570](https://github.com/gocolly/colly/pull/570) - Updated xmlquery to close a known security vulnerability by [@&#8203;ijomeli](https://github.com/ijomeli) in [#&#8203;582](https://github.com/gocolly/colly/pull/582) - Fix default User-Agent when using custom headers by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;592](https://github.com/gocolly/colly/pull/592) - Update openedx_courses.go by [@&#8203;BigManing](https://github.com/BigManing) in [#&#8203;576](https://github.com/gocolly/colly/pull/576) - Add httpBackend cache response callback by [@&#8203;slashapp](https://github.com/slashapp) in [#&#8203;574](https://github.com/gocolly/colly/pull/574) - Update README.md by [@&#8203;seversky](https://github.com/seversky) in [#&#8203;595](https://github.com/gocolly/colly/pull/595) - Change the type assersion for Attr by [@&#8203;moritamori](https://github.com/moritamori) in [#&#8203;588](https://github.com/gocolly/colly/pull/588) - Update User-Agent strings for Desktop by [@&#8203;guessi](https://github.com/guessi) in [#&#8203;605](https://github.com/gocolly/colly/pull/605) - add quote crawler using colly by [@&#8203;eval-exec](https://github.com/eval-exec) in [#&#8203;603](https://github.com/gocolly/colly/pull/603) - Fix golint issues by [@&#8203;waltton](https://github.com/waltton) in [#&#8203;620](https://github.com/gocolly/colly/pull/620) - Replace Travis with GitHub Actions by [@&#8203;alexhraber](https://github.com/alexhraber) in [#&#8203;669](https://github.com/gocolly/colly/pull/669) - Use github.com/nlnwa/whatwg-url for URL parsing by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;673](https://github.com/gocolly/colly/pull/673) - Fix Async ignoring its arguments by [@&#8203;ljbink](https://github.com/ljbink) in [#&#8203;652](https://github.com/gocolly/colly/pull/652) - fix: generated code has compile error by [@&#8203;darjun](https://github.com/darjun) in [#&#8203;629](https://github.com/gocolly/colly/pull/629) - moving filterings to one function (checkFilters) and use it in case of redirect too by [@&#8203;joelazar](https://github.com/joelazar) in [#&#8203;638](https://github.com/gocolly/colly/pull/638) - Fix Contributing.md link by [@&#8203;SaladinoBelisario](https://github.com/SaladinoBelisario) in [#&#8203;641](https://github.com/gocolly/colly/pull/641) - Work around invalid URL encoding in path by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;675](https://github.com/gocolly/colly/pull/675) - Host & Custom Header Support by [@&#8203;ErikOwen](https://github.com/ErikOwen) in [#&#8203;691](https://github.com/gocolly/colly/pull/691) - Code cleanup by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;678](https://github.com/gocolly/colly/pull/678) - Add GitHub issue templates and nudge questions to Stack Overflow by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;702](https://github.com/gocolly/colly/pull/702) - Fix redirects ignoring AllowURLRevisit=false by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;681](https://github.com/gocolly/colly/pull/681) - Bump whatwg-url version by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;713](https://github.com/gocolly/colly/pull/713) - fix some typos by [@&#8203;cuishuang](https://github.com/cuishuang) in [#&#8203;736](https://github.com/gocolly/colly/pull/736) - feat: Add MaxRequests paramete by [@&#8203;guilhem](https://github.com/guilhem) in [#&#8203;722](https://github.com/gocolly/colly/pull/722) - Update CI configuration by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;749](https://github.com/gocolly/colly/pull/749) - upgrade Cascadia version by [@&#8203;kinoute](https://github.com/kinoute) in [#&#8203;650](https://github.com/gocolly/colly/pull/650) - Bump golang.org/x/net from 0.0.0-20220114011407-0dd24b26b47d to 0.7.0 by [@&#8203;dependabot](https://github.com/dependabot) in [#&#8203;756](https://github.com/gocolly/colly/pull/756) - Support websites redirecting to the same page when AllowURLRevisit is disabled by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;763](https://github.com/gocolly/colly/pull/763) - Update User-Agent strings for 2023 by [@&#8203;guessi](https://github.com/guessi) in [#&#8203;765](https://github.com/gocolly/colly/pull/765) - Fix data races by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;771](https://github.com/gocolly/colly/pull/771) - Replaced deprecated io/ioutil package with proper implementations of … by [@&#8203;zachary-walters](https://github.com/zachary-walters) in [#&#8203;785](https://github.com/gocolly/colly/pull/785) - Bump golang.org/x/net from 0.7.0 to 0.17.0 by [@&#8203;dependabot](https://github.com/dependabot) in [#&#8203;789](https://github.com/gocolly/colly/pull/789) - put accept header before OnRequest by [@&#8203;k4lizen](https://github.com/k4lizen) in [#&#8203;786](https://github.com/gocolly/colly/pull/786) - Fix Bug: retry scrape will lost POST requestData by [@&#8203;Shinku-Chen](https://github.com/Shinku-Chen) in [#&#8203;794](https://github.com/gocolly/colly/pull/794) - scrapper -> scraper Update README.md by [@&#8203;ideabrian](https://github.com/ideabrian) in [#&#8203;809](https://github.com/gocolly/colly/pull/809) - Test on Go 1.22, drop old versions by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;810](https://github.com/gocolly/colly/pull/810) - Bump google.golang.org/protobuf from 1.24.0 to 1.33.0 by [@&#8203;dependabot](https://github.com/dependabot) in [#&#8203;807](https://github.com/gocolly/colly/pull/807) - Implement content sniffing for HTML parsing by [@&#8203;WGH-](https://github.com/WGH-) in [#&#8203;808](https://github.com/gocolly/colly/pull/808) - Add headers/ua to robots.txt request by [@&#8203;makew0rld](https://github.com/makew0rld) in [#&#8203;838](https://github.com/gocolly/colly/pull/838) - doc: mention that OnScraped executes after OnHTML and OnXML by [@&#8203;aladmit](https://github.com/aladmit) in [#&#8203;832](https://github.com/gocolly/colly/pull/832) #### New Contributors - [@&#8203;twiny](https://github.com/twiny) made their first contribution in [#&#8203;525](https://github.com/gocolly/colly/pull/525) - [@&#8203;kcq](https://github.com/kcq) made their first contribution in [#&#8203;526](https://github.com/gocolly/colly/pull/526) - [@&#8203;ysung6](https://github.com/ysung6) made their first contribution in [#&#8203;537](https://github.com/gocolly/colly/pull/537) - [@&#8203;lingt-xyz](https://github.com/lingt-xyz) made their first contribution in [#&#8203;539](https://github.com/gocolly/colly/pull/539) - [@&#8203;PalanQu](https://github.com/PalanQu) made their first contribution in [#&#8203;531](https://github.com/gocolly/colly/pull/531) - [@&#8203;dasshield](https://github.com/dasshield) made their first contribution in [#&#8203;549](https://github.com/gocolly/colly/pull/549) - [@&#8203;shubham14bajpai](https://github.com/shubham14bajpai) made their first contribution in [#&#8203;563](https://github.com/gocolly/colly/pull/563) - [@&#8203;wowinter13](https://github.com/wowinter13) made their first contribution in [#&#8203;567](https://github.com/gocolly/colly/pull/567) - [@&#8203;BigManing](https://github.com/BigManing) made their first contribution in [#&#8203;570](https://github.com/gocolly/colly/pull/570) - [@&#8203;ijomeli](https://github.com/ijomeli) made their first contribution in [#&#8203;582](https://github.com/gocolly/colly/pull/582) - [@&#8203;slashapp](https://github.com/slashapp) made their first contribution in [#&#8203;574](https://github.com/gocolly/colly/pull/574) - [@&#8203;seversky](https://github.com/seversky) made their first contribution in [#&#8203;595](https://github.com/gocolly/colly/pull/595) - [@&#8203;moritamori](https://github.com/moritamori) made their first contribution in [#&#8203;588](https://github.com/gocolly/colly/pull/588) - [@&#8203;guessi](https://github.com/guessi) made their first contribution in [#&#8203;605](https://github.com/gocolly/colly/pull/605) - [@&#8203;eval-exec](https://github.com/eval-exec) made their first contribution in [#&#8203;603](https://github.com/gocolly/colly/pull/603) - [@&#8203;waltton](https://github.com/waltton) made their first contribution in [#&#8203;620](https://github.com/gocolly/colly/pull/620) - [@&#8203;alexhraber](https://github.com/alexhraber) made their first contribution in [#&#8203;669](https://github.com/gocolly/colly/pull/669) - [@&#8203;ljbink](https://github.com/ljbink) made their first contribution in [#&#8203;652](https://github.com/gocolly/colly/pull/652) - [@&#8203;darjun](https://github.com/darjun) made their first contribution in [#&#8203;629](https://github.com/gocolly/colly/pull/629) - [@&#8203;joelazar](https://github.com/joelazar) made their first contribution in [#&#8203;638](https://github.com/gocolly/colly/pull/638) - [@&#8203;SaladinoBelisario](https://github.com/SaladinoBelisario) made their first contribution in [#&#8203;641](https://github.com/gocolly/colly/pull/641) - [@&#8203;ErikOwen](https://github.com/ErikOwen) made their first contribution in [#&#8203;691](https://github.com/gocolly/colly/pull/691) - [@&#8203;cuishuang](https://github.com/cuishuang) made their first contribution in [#&#8203;736](https://github.com/gocolly/colly/pull/736) - [@&#8203;guilhem](https://github.com/guilhem) made their first contribution in [#&#8203;722](https://github.com/gocolly/colly/pull/722) - [@&#8203;kinoute](https://github.com/kinoute) made their first contribution in [#&#8203;650](https://github.com/gocolly/colly/pull/650) - [@&#8203;dependabot](https://github.com/dependabot) made their first contribution in [#&#8203;756](https://github.com/gocolly/colly/pull/756) - [@&#8203;zachary-walters](https://github.com/zachary-walters) made their first contribution in [#&#8203;785](https://github.com/gocolly/colly/pull/785) - [@&#8203;k4lizen](https://github.com/k4lizen) made their first contribution in [#&#8203;786](https://github.com/gocolly/colly/pull/786) - [@&#8203;Shinku-Chen](https://github.com/Shinku-Chen) made their first contribution in [#&#8203;794](https://github.com/gocolly/colly/pull/794) - [@&#8203;ideabrian](https://github.com/ideabrian) made their first contribution in [#&#8203;809](https://github.com/gocolly/colly/pull/809) - [@&#8203;makew0rld](https://github.com/makew0rld) made their first contribution in [#&#8203;838](https://github.com/gocolly/colly/pull/838) - [@&#8203;aladmit](https://github.com/aladmit) made their first contribution in [#&#8203;832](https://github.com/gocolly/colly/pull/832) **Full Changelog**: <https://github.com/gocolly/colly/compare/v2.1.0...v2.2.0> ### [`v2.1.0`](https://github.com/gocolly/colly/releases/tag/v2.1.0) [Compare Source](https://github.com/gocolly/colly/compare/v2.0.1...v2.1.0) - HTTP tracing support - New callback: OnResponseHeader - Queue fixes - New collector option: Collector.CheckHead - Proxy fixes - Fixed POST revisit checking - Updated dependencies ### [`v2.0.1`](https://github.com/gocolly/colly/compare/v2.0.0...v2.0.1) [Compare Source](https://github.com/gocolly/colly/compare/v2.0.0...v2.0.1) ### [`v2.0.0`](https://github.com/gocolly/colly/releases/tag/v2.0.0) [Compare Source](https://github.com/gocolly/colly/compare/v1.2.0...v2.0.0) - Breaking change: Change Collector.RedirectHandler member to Collector.SetRedirectHandler function - Go module support - Collector.HasVisited method added to be able to check if an url has been visited - Collector.SetClient method introduced - HTMLElement.ChildTexts method added - New user agents - Multiple bugfixes </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xMTUuMTAiLCJ1cGRhdGVkSW5WZXIiOiI0NC4xMzguMCIsInRhcmdldEJyYW5jaCI6Im1hc3RlciIsImxhYmVscyI6W119-->
Update module github.com/gocolly/colly to v2
All checks were successful
Lint / Run golangci-lint (pull_request) Has been skipped
Deploy Parent Container / determine-workflow (pull_request) Successful in 2s
Deploy Parent Container / nonprod-deploy (dev) (pull_request) Has been skipped
Deploy Parent Container / prod-deploy (dev) (pull_request) Has been skipped
ef4ad272ba
All checks were successful
Lint / Run golangci-lint (pull_request) Has been skipped
Deploy Parent Container / determine-workflow (pull_request) Successful in 2s
Deploy Parent Container / nonprod-deploy (dev) (pull_request) Has been skipped
Deploy Parent Container / prod-deploy (dev) (pull_request) Has been skipped
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin renovate/github.com-gocolly-colly-2.x:renovate/github.com-gocolly-colly-2.x
git switch renovate/github.com-gocolly-colly-2.x

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch master
git merge --no-ff renovate/github.com-gocolly-colly-2.x
git switch renovate/github.com-gocolly-colly-2.x
git rebase master
git switch master
git merge --ff-only renovate/github.com-gocolly-colly-2.x
git switch renovate/github.com-gocolly-colly-2.x
git rebase master
git switch master
git merge --no-ff renovate/github.com-gocolly-colly-2.x
git switch master
git merge --squash renovate/github.com-gocolly-colly-2.x
git switch master
git merge --ff-only renovate/github.com-gocolly-colly-2.x
git switch master
git merge renovate/github.com-gocolly-colly-2.x
git push origin master
Sign in to join this conversation.
No description provided.