Scrapy 1.4 does not bring that many breathtaking new features

but quite a few handy improvements nonetheless.

Scrapy now supports anonymous FTP sessions with customizable user and
password via the new :setting:`FTP_USER` and :setting:`FTP_PASSWORD` settings.
And if you're using Twisted version 17.1.0 or above, FTP is now available
with Python 3.

There's a new :meth:`response.follow <scrapy.http.TextResponse.follow>` method
for creating requests; **it is now a recommended way to create Requests
in Scrapy spiders**. This method makes it easier to write correct
spiders; ``response.follow`` has several advantages over creating
``scrapy.Request`` objects directly:

* it handles relative URLs;
* it works properly with non-ascii URLs on non-UTF8 pages;
* in addition to absolute and relative URLs it supports Selectors;
  for ``<a>`` elements it can also extract their href values.

4 files changed, 20 insertions, 9 deletions

diff --git a/www/py-scrapy/ALTERNATIVES b/www/py-scrapy/ALTERNATIVES
new file mode 100644
index 00000000000..538b4fe96bf
--- /dev/null
+++ b/www/py-scrapy/ALTERNATIVES
@@ -0,0 +1 @@
+bin/scrapy @PREFIX@/bin/scrapy@PYVERSSUFFIX@
diff --git a/www/py-scrapy/Makefile b/www/py-scrapy/Makefile
index a44d84ba704..8e2ff2cea8b 100644
--- a/www/py-scrapy/Makefile
+++ b/www/py-scrapy/Makefile
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.2 2017/03/19 22:59:10 adam Exp $
+# $NetBSD: Makefile,v 1.3 2017/05/20 06:25:36 adam Exp $
 
-DISTNAME=	Scrapy-1.3.3
+DISTNAME=	Scrapy-1.4.0
 PKGNAME=	${PYPKGPREFIX}-${DISTNAME:tl}
 CATEGORIES=	www python
 MASTER_SITES=	${MASTER_SITE_PYPI:=S/Scrapy/}
@@ -15,5 +15,9 @@ DEPENDS+=	${PYPKGPREFIX}-parsel>=1.1.0:../../www/py-parsel
 DEPENDS+=	${PYPKGPREFIX}-pydispatcher>=2.0.5:../../devel/py-pydispatcher
 DEPENDS+=	${PYPKGPREFIX}-twisted>=13.1.0:../../net/py-twisted
 
+post-install:
+	${MV} ${DESTDIR}${PREFIX}/bin/scrapy \
+		${DESTDIR}${PREFIX}/bin/scrapy${PYVERSSUFFIX} || ${TRUE}
+
 .include "../../lang/python/egg.mk"
 .include "../../mk/bsd.pkg.mk"
diff --git a/www/py-scrapy/PLIST b/www/py-scrapy/PLIST
index 03f81ccf234..4bfb7b1f917 100644
--- a/www/py-scrapy/PLIST
+++ b/www/py-scrapy/PLIST
@@ -1,5 +1,5 @@
-@comment $NetBSD: PLIST,v 1.1 2017/02/13 21:25:33 adam Exp $
-bin/scrapy
+@comment $NetBSD: PLIST,v 1.2 2017/05/20 06:25:36 adam Exp $
+bin/scrapy${PYVERSSUFFIX}
 ${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
 ${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
 ${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt
@@ -11,6 +11,9 @@ ${PYSITELIB}/scrapy/VERSION
 ${PYSITELIB}/scrapy/__init__.py
 ${PYSITELIB}/scrapy/__init__.pyc
 ${PYSITELIB}/scrapy/__init__.pyo
+${PYSITELIB}/scrapy/__main__.py
+${PYSITELIB}/scrapy/__main__.pyc
+${PYSITELIB}/scrapy/__main__.pyo
 ${PYSITELIB}/scrapy/_monkeypatches.py
 ${PYSITELIB}/scrapy/_monkeypatches.pyc
 ${PYSITELIB}/scrapy/_monkeypatches.pyo
@@ -254,6 +257,9 @@ ${PYSITELIB}/scrapy/core/downloader/contextfactory.pyo
 ${PYSITELIB}/scrapy/core/downloader/handlers/__init__.py
 ${PYSITELIB}/scrapy/core/downloader/handlers/__init__.pyc
 ${PYSITELIB}/scrapy/core/downloader/handlers/__init__.pyo
+${PYSITELIB}/scrapy/core/downloader/handlers/datauri.py
+${PYSITELIB}/scrapy/core/downloader/handlers/datauri.pyc
+${PYSITELIB}/scrapy/core/downloader/handlers/datauri.pyo
 ${PYSITELIB}/scrapy/core/downloader/handlers/file.py
 ${PYSITELIB}/scrapy/core/downloader/handlers/file.pyc
 ${PYSITELIB}/scrapy/core/downloader/handlers/file.pyo
diff --git a/www/py-scrapy/distinfo b/www/py-scrapy/distinfo
index 91ce6b9aba1..784ff78fd3d 100644
--- a/www/py-scrapy/distinfo
+++ b/www/py-scrapy/distinfo
@@ -1,6 +1,6 @@
-$NetBSD: distinfo,v 1.2 2017/03/19 22:59:11 adam Exp $
+$NetBSD: distinfo,v 1.3 2017/05/20 06:25:36 adam Exp $
 
-SHA1 (Scrapy-1.3.3.tar.gz) = 603b258932b868ad6315e374b7b3de8c45564264
-RMD160 (Scrapy-1.3.3.tar.gz) = a23e026285640af4cf1ec29590040a67ef57326d
-SHA512 (Scrapy-1.3.3.tar.gz) = 795ac4a421be6e903e14cb9d9b242f4e6a130e3f51d43bf2a0904de0e4cadb4f54837badb63a65331ab67c4f8d6111f6fc85e6cb79acca5544db128b55ba3867
-Size (Scrapy-1.3.3.tar.gz) = 848990 bytes
+SHA1 (Scrapy-1.4.0.tar.gz) = 24222debf2e6b9220a91a56c476c208ac5ecb8e5
+RMD160 (Scrapy-1.4.0.tar.gz) = ef20b9288851962fb552c1045e297c8917a74d17
+SHA512 (Scrapy-1.4.0.tar.gz) = eedcd7003c51f45a580f160b4e5f428c01713e4ecb5b64e35570bc750fc03bef7cc991e318ef4ff9c96e12a2d21cc32d0f07ce278486ea2e65f08e53c3e4a8f1
+Size (Scrapy-1.4.0.tar.gz) = 898159 bytes