refer to apt-secure(8) in unsecure repositories warning

The manpage is also slightly updated to work better as a central hub to
push people from all angles into the right directions without writting a
book disguised as an error message.

8 files changed, 160 insertions, 46 deletions

diff --git a/apt-pkg/acquire-item.cc b/apt-pkg/acquire-item.cc
index 0c7c7c75c..5fda1439c 100644
--- a/apt-pkg/acquire-item.cc
+++ b/apt-pkg/acquire-item.cc
@@ -160,6 +160,7 @@ static bool MessageInsecureRepository(bool const isError, std::string const &msg
       _error->Warning("%s", msg.c_str());
       _error->Notice("%s", _("Data from such a repository can not be authenticated and is therefore potentially dangerous to use."));
    }
+   _error->Notice("%s", _("See apt-secure(8) manpage for repository creation and user configuration details."));
    return false;
 }
 static bool MessageInsecureRepository(bool const isError, char const * const msg, std::string const &repo)
@@ -182,7 +183,6 @@ static bool AllowInsecureRepositories(char const * const msg, std::string const
    }
 
    MessageInsecureRepository(true, msg, repo);
-   _error->Notice(_("Use --allow-insecure-repositories to force an insecure update"));
    TransactionManager->AbortTransaction();
    I->Status = pkgAcquire::Item::StatError;
    return false;
diff --git a/doc/apt-key.8.xml b/doc/apt-key.8.xml
index 1d91790ea..41628aff6 100644
--- a/doc/apt-key.8.xml
+++ b/doc/apt-key.8.xml
@@ -48,7 +48,11 @@
        &synopsis-param-filename; or if the filename is <literal>-</literal>
        from standard input.
      </para>
-
+     <para>
+     It is critical that keys added manually via <command>apt-key</command> are
+     verified to belong to the owner of the repositories they claim to be for
+     otherwise the &apt-secure; infrastructure is completely undermined.
+     </para>
      </listitem>
      </varlistentry>
 
@@ -110,10 +114,11 @@
      <varlistentry><term><option>adv</option></term>
      <listitem>
      <para>
-
-     Pass advanced options to gpg. With adv --recv-key you can download the 
-	 public key.  
-
+     Pass advanced options to gpg. With <command>adv --recv-key</command> you
+     can e.g. download key from keyservers directly into the the trusted set of
+     keys. Note that there are <emphasis>no</emphasis> checks performed, so it is
+     easy to completely undermine the &apt-secure; infrastructure if used without
+     care.
      </para>
 
      </listitem>
diff --git a/doc/apt-secure.8.xml b/doc/apt-secure.8.xml
index e343b86ea..9b4b7378d 100644
--- a/doc/apt-secure.8.xml
+++ b/doc/apt-secure.8.xml
@@ -13,7 +13,7 @@
    &apt-email;
    &apt-product;
    <!-- The last update date -->
-   <date>2012-06-09T00:00:00Z</date>
+   <date>2015-10-14T00:00:00Z</date>
  </refentryinfo>
 
  <refmeta>
@@ -45,32 +45,38 @@
 
  <refsect1><title>Description</title>
    <para>
-   Starting with version 0.6, <command>apt</command> contains code
-   that does signature checking of the Release file for all
-   archives. This ensures that packages in the archive can't be
-   modified by people who have no access to the Release file signing
-   key.
+   Starting with version 0.6, <command>APT</command> contains code that does
+   signature checking of the Release file for all repositories. This ensures
+   that data like packages in the archive can't be modified by people who
+   have no access to the Release file signing key.
    </para>
 
    <para>
-   If a package comes from a archive without a signature, or with a
-   signature that apt does not have a key for, that package is
-   considered untrusted, and installing it will result in a big
-   warning. <command>apt-get</command> will currently only warn
-   for unsigned archives; future releases might force all sources
-   to be verified before downloading packages from them.
+   If an archive doesn't have a signed Release file or no Release file at all
+   current APT versions will raise a warning in <command>update</command>
+   operations and frontends like <command>apt-get</command> will require
+   explicit confirmation if an installation request includes a package from
+   such an unauthenticated archive.
    </para>
 
    <para>
-   The package frontends &apt-get;, &aptitude; and &synaptic; support this new
-   authentication feature.
+   In the future APT will refuse to work with unauthenticated repositories by
+   default until support for them is removed entirely. Users have the option to
+   opt-in to this behavior already by setting the configuration option
+   <option>Acquire::AllowInsecureRepositories</option> to <literal>false</literal>.
+   </para>
+
+   <para>
+   Note: All APT-based package management frontends like &apt-get;, &aptitude;
+   and &synaptic; support this authentication feature, so this manpage uses
+   <literal>APT</literal> to refer to them all for simplicity only.
    </para>
 </refsect1>
 
- <refsect1><title>Trusted archives</title> 
+ <refsect1><title>Trusted repositories</title>
 
-   <para> 
-   The chain of trust from an apt archive to the end user is made up of
+   <para>
+   The chain of trust from an APT archive to the end user is made up of
    several steps. <command>apt-secure</command> is the last step in
    this chain; trusting an archive does not mean that you trust its
    packages not to contain malicious code, but means that you
@@ -85,13 +91,14 @@
    devscripts packages respectively).</para>
 
    <para>
-   The chain of trust in Debian starts when a maintainer uploads a new
+   The chain of trust in Debian e.g. starts when a maintainer uploads a new
    package or a new version of a package to the Debian archive. In
    order to become effective, this upload needs to be signed by a key
-   contained in the Debian Maintainers keyring (available in
+   contained in one of the Debian package maintainers keyrings (available in
    the debian-keyring package). Maintainers' keys are signed by
    other maintainers following pre-established procedures to
-   ensure the identity of the key holder.
+   ensure the identity of the key holder. Similar procedures exist in all
+   Debian-based distributions.
    </para>
 
    <para>
@@ -132,20 +139,23 @@
     </itemizedlist>
 
    <para>However, it does not defend against a compromise of the
-   Debian master server itself (which signs the packages) or against a
+   master server itself (which signs the packages) or against a
    compromise of the key used to sign the Release files. In any case,
    this mechanism can complement a per-package signature.</para>
 </refsect1>
 
  <refsect1><title>User configuration</title>
    <para>
-   <command>apt-key</command> is the program that manages the list
-   of keys used by apt. It can be used to add or remove keys, although
-   an installation of this release will automatically contain the
-   default Debian archive signing keys used in the Debian package
-   repositories.
-   </para>
-   <para>
+   <command>apt-key</command> is the program that manages the list of keys used
+   by APT to trust repositories. It can be used to add or remove keys as well
+   as list the trusted keys. Limiting which key(s) are able to sign which archive
+   is possible via the <option>Signed-By</option> in &sources-list;.
+   </para><para>
+   Note that a default installation already contains all keys to securily
+   acquire packages from the default repositories, so fiddling with
+   <command>apt-key</command> is only needed if third-party repositories are
+   added.
+   </para><para>
    In order to add a new key you need to first download it
    (you should make sure you are using a trusted communication channel
    when retrieving it), add it with <command>apt-key</command> and
@@ -171,10 +181,21 @@
       <command>gpg --clearsign -o InRelease Release</command> and
       <command>gpg -abs -o Release.gpg Release</command>.</para></listitem>
 
-      <listitem><para><emphasis>Publish the key fingerprint</emphasis>,
-      that way your users will know what key they need to import in
-      order to authenticate the files in the
-      archive.</para></listitem>
+      <listitem><para>
+      <emphasis>Publish the key fingerprint</emphasis>, that way your users
+      will know what key they need to import in order to authenticate the files
+      in the archive. It is best to ship your key in its own keyring package
+      like &keyring-distro; does with &keyring-package; to be able to
+      distribute updates and key transitions automatically later.
+      </para></listitem>
+
+      <listitem><para>
+      <emphasis>Provide instructions on how to add your archive and key</emphasis>.
+      If your users can't acquire your key securily the chain of trust described above is broken.
+      How you can help users add your key depends on your archive and target audience ranging
+      from having your keyring package included in another archive users already have configured
+      (like the default repositories of their distribution) to leverage the web of trust.
+      </para></listitem>
 
     </itemizedlist>
 
@@ -192,7 +213,7 @@
 
 <para>For more background information you might want to review the
 <ulink
-url="http://www.debian.org/doc/manuals/securing-debian-howto/ch7">Debian
+url="https://www.debian.org/doc/manuals/securing-debian-howto/ch7">Debian
 Security Infrastructure</ulink> chapter of the Securing Debian Manual
 (available also in the harden-doc package) and the
 <ulink url="http://www.cryptnet.net/fdp/crypto/strong_distro.html"
diff --git a/test/integration/framework b/test/integration/framework
index b4220c8b5..8b85cb71e 100644
--- a/test/integration/framework
+++ b/test/integration/framework
@@ -1661,7 +1661,7 @@ testfailuremsg() {
 	testfailure "$@"
 	msgtest 'Check that the output of the previous failed command has expected' 'failures and warnings'
 	local COMPAREFILE="${TMPWORKINGDIRECTORY}/rootdir/tmp/testfailuremsg.comparefile"
-	grep '^\(W\|E\):' "${TMPWORKINGDIRECTORY}/rootdir/tmp/testfailure.output" > "$COMPAREFILE" 2>&1 || true
+	grep '^\(W\|E\|N\):' "${TMPWORKINGDIRECTORY}/rootdir/tmp/testfailure.output" > "$COMPAREFILE" 2>&1 || true
 	testoutputequal "$COMPAREFILE" echo "$CMP"
 	msggroup
 }
@@ -1672,7 +1672,7 @@ testwarningmsg() {
 	testwarning "$@"
 	msgtest 'Check that the output of the previous warned command has expected' 'warnings'
 	local COMPAREFILE="${TMPWORKINGDIRECTORY}/rootdir/tmp/testwarningmsg.comparefile"
-	grep '^\(W\|E\):' "${TMPWORKINGDIRECTORY}/rootdir/tmp/testwarning.output" > "$COMPAREFILE" 2>&1 || true
+	grep '^\(W\|E\|N\):' "${TMPWORKINGDIRECTORY}/rootdir/tmp/testwarning.output" > "$COMPAREFILE" 2>&1 || true
 	testoutputequal "$COMPAREFILE" echo "$CMP"
 	msggroup
 }
diff --git a/test/integration/test-apt-get-update-unauth-warning b/test/integration/test-apt-get-update-unauth-warning
index fad1cf627..f1515a9c8 100755
--- a/test/integration/test-apt-get-update-unauth-warning
+++ b/test/integration/test-apt-get-update-unauth-warning
@@ -29,7 +29,7 @@ Err:2 file:$APTARCHIVE unstable Release
 Reading package lists...
 E: The repository 'file:$APTARCHIVE unstable Release' does not have a Release file.
 N: Updating such a repository securily is impossible and therefore disabled by default.
-N: Use --allow-insecure-repositories to force an insecure update" aptget update --no-allow-insecure-repositories -q=0
+N: See apt-secure(8) manpage for repository creation and user configuration details." aptget update --no-allow-insecure-repositories -q=0
 
 # no package foo
 testsuccessequal 'Listing...' apt list foo
@@ -80,7 +80,8 @@ Get:4 file:$APTARCHIVE unstable/main i386 Packages [$(filesize 'Packages') B]
 Get:5 file:$APTARCHIVE unstable/main Translation-en [$(filesize 'Translations') B]
 Reading package lists...
 W: The repository 'file:$APTARCHIVE unstable Release' does not have a Release file.
-N: Data from such a repository can not be authenticated and is therefore potentially dangerous to use." aptget update --allow-insecure-repositories -q=0
+N: Data from such a repository can not be authenticated and is therefore potentially dangerous to use.
+N: See apt-secure(8) manpage for repository creation and user configuration details." aptget update --allow-insecure-repositories -q=0
 # ensure we can not install the package
 testfailureequal "WARNING: The following packages cannot be authenticated!
   foo
diff --git a/test/integration/test-apt-update-failure-propagation b/test/integration/test-apt-update-failure-propagation
new file mode 100755
index 000000000..713f09db7
--- /dev/null
+++ b/test/integration/test-apt-update-failure-propagation
@@ -0,0 +1,82 @@
+#!/bin/sh
+set -e
+
+TESTDIR="$(readlink -f "$(dirname "$0")")"
+. "$TESTDIR/framework"
+setupenvironment
+configarchitecture 'amd64'
+
+buildsimplenativepackage 'foo' 'all' '1' 'stable'
+buildsimplenativepackage 'foo' 'all' '2' 'sid'
+setupaptarchive --no-update
+
+NEWMETHODS="$(readlink -f rootdir)/usr/lib/apt/methods"
+OLDMETHODS="$(readlink -f rootdir/usr/lib/apt/methods)"
+rm "$NEWMETHODS"
+mkdir "$NEWMETHODS"
+backupIFS="$IFS"
+IFS="$(printf "\n\b")"
+for METH in $(find "$OLDMETHODS" ! -type d); do
+	ln -s "$OLDMETHODS/$(basename "$METH")" "$NEWMETHODS"
+done
+IFS="$backupIFS"
+
+changetohttpswebserver
+for FILE in rootdir/etc/apt/sources.list.d/*-sid-* ; do
+	sed -i -e 's#https:#http:#' -e "s#:${APTHTTPSPORT}/#:${APTHTTPPORT}/#" "$FILE"
+done
+
+pretest() {
+	rm -rf rootdir/var/lib/apt/lists
+	testsuccessequal 'N: Unable to locate package foo' aptcache policy foo -q=0
+}
+pretest
+testsuccess aptget update
+testsuccessequal "foo:
+  Installed: (none)
+  Candidate: 2
+  Version table:
+     2 500
+        500 http://localhost:${APTHTTPPORT} sid/main amd64 Packages
+     1 500
+        500 https://localhost:${APTHTTPSPORT} stable/main amd64 Packages" aptcache policy foo
+
+pretest
+mv aptarchive/dists/stable aptarchive/dists/stable.good
+testfailuremsg "E: The repository 'https://localhost:${APTHTTPSPORT} stable Release' does not have a Release file." aptget update
+testfailureequal "Hit:1 http://localhost:${APTHTTPPORT} sid InRelease
+Ign:2 https://localhost:${APTHTTPSPORT} stable InRelease
+  404  Not Found
+Err:3 https://localhost:${APTHTTPSPORT} stable Release
+  404  Not Found
+Reading package lists...
+E: The repository 'https://localhost:${APTHTTPSPORT} stable Release' does not have a Release file.
+N: Updating such a repository securily is impossible and therefore disabled by default.
+N: See apt-secure(8) manpage for repository creation and user configuration details." aptget update -q=0
+mv aptarchive/dists/stable.good aptarchive/dists/stable
+posttest() {
+	testsuccessequal "foo:
+  Installed: (none)
+  Candidate: 2
+  Version table:
+     2 500
+        500 http://localhost:${APTHTTPPORT} sid/main amd64 Packages" aptcache policy foo
+}
+posttest
+
+pretest
+rm "${NEWMETHODS}/https"
+testfailuremsg "E: The method driver ${TMPWORKINGDIRECTORY}/rootdir/usr/lib/apt/methods/https could not be found.
+W: Failed to fetch https://localhost:${APTHTTPSPORT}/dists/stable/InRelease  
+E: Some index files failed to download. They have been ignored, or old ones used instead." aptget update
+posttest
+
+ln -s "$OLDMETHODS/https" "$NEWMETHODS"
+pretest
+for FILE in rootdir/etc/apt/sources.list.d/*-stable-* ; do
+	# lets see how many testservers run also Doom
+	sed -i -e "s#:${APTHTTPSPORT}/#:666/#" "$FILE"
+done
+testwarningmsg "W: Failed to fetch https://localhost:666/dists/stable/InRelease  Failed to connect to localhost port 666: Connection refused
+W: Some index files failed to download. They have been ignored, or old ones used instead." aptget update
+posttest
diff --git a/test/integration/test-apt-update-ims b/test/integration/test-apt-update-ims
index 3a66a546f..4c25186f5 100755
--- a/test/integration/test-apt-update-ims
+++ b/test/integration/test-apt-update-ims
@@ -81,7 +81,8 @@ Ign:3 http://localhost:${APTHTTPPORT} unstable Release.gpg
   404  Not Found
 Reading package lists...
 W: The repository 'http://localhost:${APTHTTPPORT} unstable Release' is not signed.
-N: Data from such a repository can not be authenticated and is therefore potentially dangerous to use."
+N: Data from such a repository can not be authenticated and is therefore potentially dangerous to use.
+N: See apt-secure(8) manpage for repository creation and user configuration details."
 find aptarchive -name 'Release.gpg' -delete
 echo 'Acquire::GzipIndexes "0";' > rootdir/etc/apt/apt.conf.d/02compressindex
 runtest 'warning'
@@ -126,6 +127,7 @@ Ign:3 http://localhost:${APTHTTPPORT} unstable Release.gpg
 Reading package lists...
 W: The repository 'http://localhost:${APTHTTPPORT} unstable Release' is not signed.
 N: Data from such a repository can not be authenticated and is therefore potentially dangerous to use.
+N: See apt-secure(8) manpage for repository creation and user configuration details.
 E: Release file for http://localhost:${APTHTTPPORT}/dists/unstable/Release is expired (invalid since). Updates for this repository will not be applied."
 find aptarchive -name 'Release.gpg' -delete
 echo 'Acquire::GzipIndexes "0";' > rootdir/etc/apt/apt.conf.d/02compressindex
@@ -162,7 +164,8 @@ Hit:4 http://localhost:${APTHTTPPORT} unstable/main amd64 Packages
 Hit:5 http://localhost:${APTHTTPPORT} unstable/main Translation-en
 Reading package lists...
 W: The repository 'http://localhost:${APTHTTPPORT} unstable Release' does not have a Release file.
-N: Data from such a repository can not be authenticated and is therefore potentially dangerous to use."
+N: Data from such a repository can not be authenticated and is therefore potentially dangerous to use.
+N: See apt-secure(8) manpage for repository creation and user configuration details."
 find aptarchive -name '*Release*' -delete
 echo 'Acquire::GzipIndexes "0";
 Acquire::PDiffs "0";' > rootdir/etc/apt/apt.conf.d/02compressindex
diff --git a/test/integration/test-apt-update-nofallback b/test/integration/test-apt-update-nofallback
index e82483da3..1b23d4f11 100755
--- a/test/integration/test-apt-update-nofallback
+++ b/test/integration/test-apt-update-nofallback
@@ -33,7 +33,9 @@ EOF
 
 assert_update_is_refused_and_last_good_state_used()
 {
-    testfailuremsg "E: The repository 'file:${APTARCHIVE} unstable Release' is no longer signed." aptget update
+    testfailuremsg "E: The repository 'file:${APTARCHIVE} unstable Release' is no longer signed.
+N: Updating such a repository securily is impossible and therefore disabled by default.
+N: See apt-secure(8) manpage for repository creation and user configuration details." aptget update -q=0
 
     assert_repo_is_intact
 }