* merged from main

Patches applied:

 * apt@packages.debian.org/apt--main--0--patch-106
   Restore lost changelog entries

 * apt@packages.debian.org/apt--main--0--patch-107
   Merge michael.vogt@ubuntu.com--2005/apt--progress-reporting--0

 * apt@packages.debian.org/apt--main--0--patch-108
   Merge michael.vogt@ubuntu.com--2005/apt--progress-reporting--0

 * apt@packages.debian.org/apt--main--0--patch-109
   Merge michael.vogt@ubuntu.com--2005/apt--progress-reporting--0

 * apt@packages.debian.org/apt--main--0--patch-110
   Merge michael.vogt@ubuntu.com--2005/apt--progress-reporting--0

 * michael.vogt@ubuntu.com--2005/apt--fixes--0--patch-5
   * merged with apt--main--0

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--base-0
   tag of apt@packages.debian.org/apt--main--0--patch-85

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-1
   * inital proof of concept code, understands what dpkg tells it already

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-2
   * progress reporting works now

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-3
   * added "APT::Status-Fd" variable

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-4
   * do i18n now too

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-5
   * define N_(x) if it is not defined already

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-6
   * PackageManager::DoInstall(int status_fd) added (does not break the ABI)

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-7
   * merged with apt--fixes--0 to make it build again

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-8
   * added support for "error" and "conffile-prompt" messages from dpkg

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-9
   merge with main

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-10
   * use sizeof() for all snprintf() uses; fix a potential line break problem in the status reading code; changed the N_() to _() calls

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-11
   * added APT::KeepFDs configuration list for file descriptors that apt should leave open (needed for various frontends like debconf, synaptic)

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-12
   * fixed a API breakage

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-13
   * doc added, should be releasable now

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-14
   * merged with apt--main--0

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-15
   * more source comments, added Debug::DpkgPM debug code to inspect the dpkg<->apt communication, broke the abi (ok with matt)

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-16
   * the progress reporting has it's own "Debug::pkgDPkgProgressReporting" debug variable now

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-17
   * merged PackageOps and TranslatedPackageOps into a single Map with the new DpkgState struct

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-18
   * clear the APT::Keep-Fds configuration when it's no longer needed

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-19
   * rewrote the reading from dpkg so that it never blocks

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-20
   * merged the two status arrays into one

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-21
   * added support for download progress reporting too (for Kamion and base-config)

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-22
   * ABI break; added Configuration::Clear(string List, {int,string} value) added (to remove a single Value from a list); test/conf_clear.cc added

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-23
   * remvoed a debug string

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-24
   * soname changed, fixed a bug in the parsing code when dpkg send the same state more than once (at the end)

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-25
   * merged with apt@packages.debian.org/apt--main--0, added changelog entry for the 0.6.40.1 upload

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-26
   * fix a bug when out-of-order states are send from dpkg 

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-27
   * changelog update

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-28
   * a real changelog entry now

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-29
   * changelog finalized

 * michael.vogt@ubuntu.com--2005/apt--progress-reporting--0--patch-30
   * propper (and sane) support for pmerror and pmconffile added

21 files changed, 681 insertions, 45 deletions

diff --git a/README.progress-reporting b/README.progress-reporting
new file mode 100644
index 000000000..73fbd8c08
--- /dev/null
+++ b/README.progress-reporting
@@ -0,0 +1,62 @@
+Install-progress reporting 
+--------------------------
+
+If the apt options: "APT::Status-Fd" is set, apt will send status
+reports to that fd. The status information is seperated with a ':', 
+there are the following status conditions:
+
+status = {"pmstatus", "dlstatus", "conffile-prompt", "error" } 
+
+The reason for using a fd instead of a OpProgress class is that many
+apt frontend fork a (vte) terminal for the actual installation.
+
+The reason to do the mapping and l10n of the dpkg states to human
+readable (and translatable) strings is that this way the translation
+needs to be done only once for all frontends.
+
+
+pmstatus
+--------
+Status of the package manager (dpkg). This is send when packages
+are installed/removed.
+pmstatus:pkgname:TotalPercentage:action-description
+
+PkgName = the name of the package
+TotalPercentage = the total progress between [0..100] 
+description = a i18ned human readable decription of the current action
+
+
+Example:
+# ./apt-get install -o APT::Status-Fd=2 3dchess >/dev/null
+pmstatus:3dchess:20:Preparing 3dchess
+pmstatus:3dchess:40:Unpacking 3dchess
+pmstatus:3dchess:60:Preparing to configure 3dchess
+pmstatus:3dchess:80:Configuring 3dchess
+pmstatus:3dchess:100:Installed 3dchess
+
+pmerror
+-------
+pmerror:deb:TotalPercentage:error string
+
+Example:
+pmerror: /var/cache/apt/archives/krecipes_0.8.1-0ubuntu1_i386.deb : 75% : trying to overwrite `/usr/share/doc/kde/HTML/en/krecipes/krectip.png', which is also in package krecipes-data 
+
+
+pmconffile
+----------
+pmconffile:conffile:percent:'current-conffile' 'new-conffile' useredited distedited
+
+
+
+dlstatus
+--------
+dlstatus:AlreadDownloaded:TotalPercentage:action-description
+
+AlreadyDownloaded = the number of already downloaded packages
+TotalPercentage = the total progress between [0..100] 
+description = a i18ned human readable decription of the current action
+
+Example:
+dlstatus:1:9.05654:Downloading file 1 of 3 (4m40s remaining)
+dlstatus:1:9.46357:Downloading file 1 of 3 (4m39s remaining)
+dlstatus:1:9.61022:Downloading file 1 of 3 (4m38s remaining)
\ No newline at end of file
diff --git a/apt-pkg/acquire.cc b/apt-pkg/acquire.cc
index 70dce4f54..62209e65b 100644
--- a/apt-pkg/acquire.cc
+++ b/apt-pkg/acquire.cc
@@ -26,6 +26,7 @@
 #include <apti18n.h>
 
 #include <iostream>
+#include <sstream>
     
 #include <dirent.h>
 #include <sys/time.h>
@@ -803,6 +804,26 @@ bool pkgAcquireStatus::Pulse(pkgAcquire *Owner)
       Time = NewTime;
    }
 
+   int fd = _config->FindI("APT::Status-Fd",-1);
+   if(fd > 0) 
+   {
+      ostringstream status;
+
+      char msg[200];
+      long i = CurrentItems < TotalItems ? CurrentItems + 1 : CurrentItems;
+      unsigned long ETA =
+	 (unsigned long)((TotalBytes - CurrentBytes) / CurrentCPS);
+
+      snprintf(msg,sizeof(msg), _("Downloading file %li of %li (%s remaining)"), i, TotalItems, TimeToStr(ETA).c_str());
+
+      // build the status str
+      status << "dlstatus:" << i
+	     << ":"  << (CurrentBytes/float(TotalBytes)*100.0) 
+	     << ":" << msg 
+	     << endl;
+      write(fd, status.str().c_str(), status.str().size());
+   }
+
    return true;
 }
 									/*}}}*/
diff --git a/apt-pkg/contrib/configuration.cc b/apt-pkg/contrib/configuration.cc
index 69f8d1dca..09e454be9 100644
--- a/apt-pkg/contrib/configuration.cc
+++ b/apt-pkg/contrib/configuration.cc
@@ -327,15 +327,56 @@ void Configuration::Set(const char *Name,int Value)
    Itm->Value = S;
 }
 									/*}}}*/
+// Configuration::Clear - Clear an single value from a list	        /*{{{*/
+// ---------------------------------------------------------------------
+/* */
+void Configuration::Clear(string Name, int Value)
+{
+   char S[300];
+   snprintf(S,sizeof(S),"%i",Value);
+   Clear(Name, S);
+}
+									/*}}}*/
+// Configuration::Clear - Clear an single value from a list	        /*{{{*/
+// ---------------------------------------------------------------------
+/* */
+void Configuration::Clear(string Name, string Value)
+{
+   Item *Top = Lookup(Name.c_str(),false);
+   if (Top == 0 || Top->Child == 0)
+      return;
+
+   Item *Tmp, *Prev, *I;
+   Prev = I = Top->Child;
+
+   while(I != NULL)
+   {
+      if(I->Value == Value)
+      {
+	 Tmp = I;
+	 // was first element, point parent to new first element
+	 if(Top->Child == Tmp)
+	    Top->Child = I->Next;
+	 I = I->Next;
+	 Prev->Next = I;
+	 delete Tmp;
+      } else {
+	 Prev = I;
+	 I = I->Next;
+      }
+   }
+     
+}
+									/*}}}*/
 // Configuration::Clear - Clear an entire tree				/*{{{*/
 // ---------------------------------------------------------------------
 /* */
 void Configuration::Clear(string Name)
 {
    Item *Top = Lookup(Name.c_str(),false);
-   if (Top == 0)
+   if (Top == 0) 
       return;
-   
+
    Top->Value = string();
    Item *Stop = Top;
    Top = Top->Child;
diff --git a/apt-pkg/contrib/configuration.h b/apt-pkg/contrib/configuration.h
index 0ed8f59d3..789bc82cf 100644
--- a/apt-pkg/contrib/configuration.h
+++ b/apt-pkg/contrib/configuration.h
@@ -87,8 +87,13 @@ class Configuration
    bool Exists(const char *Name) const;
    bool ExistsAny(const char *Name) const;
 
+   // clear a whole tree
    void Clear(string Name);
-   
+
+   // remove a certain value from a list (e.g. the list of "APT::Keep-Fds")
+   void Clear(string List, string Value);
+   void Clear(string List, int Value);
+
    inline const Item *Tree(const char *Name) const {return Lookup(Name);};
 
    inline void Dump() { Dump(std::clog); };
diff --git a/apt-pkg/contrib/fileutl.cc b/apt-pkg/contrib/fileutl.cc
index 0ce0c9b9d..9fd71728e 100644
--- a/apt-pkg/contrib/fileutl.cc
+++ b/apt-pkg/contrib/fileutl.cc
@@ -20,6 +20,7 @@
 #include <apt-pkg/fileutl.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/sptr.h>
+#include <apt-pkg/configuration.h>
 
 #include <apti18n.h>
 
@@ -32,6 +33,7 @@
 #include <sys/wait.h>
 #include <signal.h>
 #include <errno.h>
+#include <set>
 									/*}}}*/
 
 using namespace std;
@@ -306,7 +308,7 @@ bool WaitFd(int Fd,bool write,unsigned long timeout)
 /* This is used if you want to cleanse the environment for the forked 
    child, it fixes up the important signals and nukes all of the fds,
    otherwise acts like normal fork. */
-pid_t ExecFork(int dontCloseThisFd)
+pid_t ExecFork()
 {
    // Fork off the process
    pid_t Process = fork();
@@ -326,11 +328,27 @@ pid_t ExecFork(int dontCloseThisFd)
       signal(SIGWINCH,SIG_DFL);
       signal(SIGCONT,SIG_DFL);
       signal(SIGTSTP,SIG_DFL);
-      
+
+      set<int> KeepFDs;
+      Configuration::Item const *Opts = _config->Tree("APT::Keep-Fds");
+      if (Opts != 0 && Opts->Child != 0)
+      {
+	 Opts = Opts->Child;
+	 for (; Opts != 0; Opts = Opts->Next)
+	 {
+	    if (Opts->Value.empty() == true)
+	       continue;
+	    int fd = atoi(Opts->Value.c_str());
+	    KeepFDs.insert(fd);
+	 }
+      }
+
       // Close all of our FDs - just in case
       for (int K = 3; K != 40; K++)
-	 if(K != dontCloseThisFd)
+      {
+	 if(KeepFDs.find(K) == KeepFDs.end())
 	    fcntl(K,F_SETFD,FD_CLOEXEC);
+      }
    }
    
    return Process;
diff --git a/apt-pkg/contrib/fileutl.h b/apt-pkg/contrib/fileutl.h
index 4716e261e..041aa3309 100644
--- a/apt-pkg/contrib/fileutl.h
+++ b/apt-pkg/contrib/fileutl.h
@@ -87,7 +87,7 @@ string SafeGetCWD();
 void SetCloseExec(int Fd,bool Close);
 void SetNonBlock(int Fd,bool Block);
 bool WaitFd(int Fd,bool write = false,unsigned long timeout = 0);
-pid_t ExecFork(int dontCloseThisFd=-1);
+pid_t ExecFork();
 bool ExecWait(pid_t Pid,const char *Name,bool Reap = false);
 
 // File string manipulators
diff --git a/apt-pkg/deb/dpkgpm.cc b/apt-pkg/deb/dpkgpm.cc
index 61c48dcbb..fe8fbca74 100644
--- a/apt-pkg/deb/dpkgpm.cc
+++ b/apt-pkg/deb/dpkgpm.cc
@@ -25,7 +25,11 @@
 #include <signal.h>
 #include <errno.h>
 #include <stdio.h>
-#include <iostream>
+#include <sstream>
+#include <map>
+
+#include <config.h>
+#include <apti18n.h>
 									/*}}}*/
 
 using namespace std;
@@ -325,8 +329,14 @@ bool pkgDPkgPM::RunScriptsWithPkgs(const char *Cnf)
 									/*}}}*/
 // DPkgPM::Go - Run the sequence					/*{{{*/
 // ---------------------------------------------------------------------
-/* This globs the operations and calls dpkg */
-bool pkgDPkgPM::Go(int status_fd)
+/* This globs the operations and calls dpkg 
+ *   
+ * If it is called with "OutStatusFd" set to a valid file descriptor
+ * apt will report the install progress over this fd. It maps the
+ * dpkg states a package goes through to human readable (and i10n-able)
+ * names and calculates a percentage for each step.
+*/
+bool pkgDPkgPM::Go(int OutStatusFd)
 {
    unsigned int MaxArgs = _config->FindI("Dpkg::MaxArgs",8*1024);   
    unsigned int MaxArgBytes = _config->FindI("Dpkg::MaxArgBytes",32*1024);
@@ -336,7 +346,66 @@ bool pkgDPkgPM::Go(int status_fd)
 
    if (RunScriptsWithPkgs("DPkg::Pre-Install-Pkgs") == false)
       return false;
+   
+   // prepare the progress reporting 
+   int Done = 0;
+   int Total = 0;
+   // map the dpkg states to the operations that are performed
+   // (this is sorted in the same way as Item::Ops)
+   static const struct DpkgState DpkgStatesOpMap[][5] = {
+      // Install operation
+      { 
+	 {"half-installed", _("Preparing %s")}, 
+	 {"unpacked", _("Unpacking %s") }, 
+	 {NULL, NULL}
+      },
+      // Configure operation
+      { 
+	 {"unpacked",_("Preparing to configure %s") },
+	 {"half-configured", _("Configuring %s") },
+	 { "installed", _("Installed %s")},
+	 {NULL, NULL}
+      },
+      // Remove operation
+      { 
+	 {"half-configured", _("Preparing for removal of %s")},
+	 {"half-installed", _("Removing %s")},
+	 {"config-files",  _("Removed %s")},
+	 {NULL, NULL}
+      },
+      // Purge operation
+      { 
+	 {"config-files", _("Preparing for remove with config %s")},
+	 {"not-installed", _("Removed with config %s")},
+	 {NULL, NULL}
+      },
+   };
 
+   // the dpkg states that the pkg will run through, the string is 
+   // the package, the vector contains the dpkg states that the package
+   // will go through
+   map<string,vector<struct DpkgState> > PackageOps;
+   // the dpkg states that are already done; the string is the package
+   // the int is the state that is already done (e.g. a package that is
+   // going to be install is already in state "half-installed")
+   map<string,int> PackageOpsDone;
+
+   // init the PackageOps map, go over the list of packages that
+   // that will be [installed|configured|removed|purged] and add
+   // them to the PackageOps map (the dpkg states it goes through)
+   // and the PackageOpsTranslations (human readable strings)
+   for (vector<Item>::iterator I = List.begin(); I != List.end();I++)
+   {
+      string name = (*I).Pkg.Name();
+      PackageOpsDone[name] = 0;
+      for(int i=0; (DpkgStatesOpMap[(*I).Op][i]).state != NULL;  i++) 
+      {
+	 PackageOps[name].push_back(DpkgStatesOpMap[(*I).Op][i]);
+	 Total++;
+      }
+   }   
+
+   // this loop is runs once per operation
    for (vector<Item>::iterator I = List.begin(); I != List.end();)
    {
       vector<Item>::iterator J = I;
@@ -367,16 +436,15 @@ bool pkgDPkgPM::Go(int status_fd)
 	 }	 
       }
       
-      // if we got a status_fd argument, we pass it to apt
       char status_fd_buf[20];
-      if(status_fd > 0) 
-      {
-	 Args[n++] = "--status-fd";
-	 Size += strlen(Args[n-1]);
-	 snprintf(status_fd_buf,20,"%i",status_fd);
-	 Args[n++] = status_fd_buf;
-	 Size += strlen(Args[n-1]);
-      } 
+      int fd[2];
+      pipe(fd);
+      
+      Args[n++] = "--status-fd";
+      Size += strlen(Args[n-1]);
+      snprintf(status_fd_buf,sizeof(status_fd_buf),"%i", fd[1]);
+      Args[n++] = status_fd_buf;
+      Size += strlen(Args[n-1]);
 
       switch (I->Op)
       {
@@ -449,17 +517,17 @@ bool pkgDPkgPM::Go(int status_fd)
 	 it doesn't die but we do! So we must also ignore it */
       sighandler_t old_SIGQUIT = signal(SIGQUIT,SIG_IGN);
       sighandler_t old_SIGINT = signal(SIGINT,SIG_IGN);
-		     
-      // Fork dpkg
+	
+       // Fork dpkg
       pid_t Child;
-      if(status_fd > 0)
-	 Child = ExecFork(status_fd);
-      else
-	 Child = ExecFork();
+      _config->Set("APT::Keep-Fds::",fd[1]);
+      Child = ExecFork();
             
       // This is the child
       if (Child == 0)
       {
+	 close(fd[0]); // close the read end of the pipe
+	 
 	 if (chdir(_config->FindDir("DPkg::Run-Directory","/").c_str()) != 0)
 	    _exit(100);
 	 
@@ -487,19 +555,141 @@ bool pkgDPkgPM::Go(int status_fd)
 	 _exit(100);
       }      
 
+      // clear the Keep-Fd again
+      _config->Clear("APT::Keep-Fds",fd[1]);
+
       // Wait for dpkg
       int Status = 0;
-      while (waitpid(Child,&Status,0) != Child)
-      {
-	 if (errno == EINTR)
+
+      // we read from dpkg here
+      int _dpkgin = fd[0];
+      fcntl(_dpkgin, F_SETFL, O_NONBLOCK);
+      close(fd[1]);                        // close the write end of the pipe
+
+      // the read buffers for the communication with dpkg
+      char line[1024] = {0,};
+      char buf[2] = {0,0};
+      
+      // the result of the waitpid call
+      int res;
+
+      while ((res=waitpid(Child,&Status, WNOHANG)) != Child) {
+	 if(res < 0) {
+	    // FIXME: move this to a function or something, looks ugly here
+	    // error handling, waitpid returned -1
+	    if (errno == EINTR)
+	       continue;
+	    RunScripts("DPkg::Post-Invoke");
+
+	    // Restore sig int/quit
+	    signal(SIGQUIT,old_SIGQUIT);
+	    signal(SIGINT,old_SIGINT);
+	    return _error->Errno("waitpid","Couldn't wait for subprocess");
+	 }
+	 
+	 // read a single char, make sure that the read can't block 
+	 // (otherwise we may leave zombies)
+         int len = read(_dpkgin, buf, 1);
+
+	 // nothing to read, wait a bit for more
+	 if(len <= 0)
+	 {
+	    usleep(1000);
 	    continue;
-	 RunScripts("DPkg::Post-Invoke");
+	 }
+	 
+	 // sanity check (should never happen)
+	 if(strlen(line) >= sizeof(line)-10)
+	 {
+	    _error->Error("got a overlong line from dpkg: '%s'",line);
+	    line[0]=0;
+	 }
+	 // append to line, check if we got a complete line
+	 strcat(line, buf);
+	 if(buf[0] != '\n')
+	    continue;
+
+	 if (_config->FindB("Debug::pkgDPkgProgressReporting",false) == true)
+	    std::clog << "got from dpkg '" << line << "'" << std::endl;
+
+	 // the status we output
+	 ostringstream status;
+
+	 /* dpkg sends strings like this:
+	    'status:   <pkg>:  <pkg  qstate>'
+	    errors look like this:
+	    'status: /var/cache/apt/archives/krecipes_0.8.1-0ubuntu1_i386.deb : error : trying to overwrite `/usr/share/doc/kde/HTML/en/krecipes/krectip.png', which is also in package krecipes-data 
+	    and conffile-prompt like this
+	    'status: conffile-prompt: conffile : 'current-conffile' 'new-conffile' useredited distedited
+	    
+	 */
+	 char* list[4];
+	 TokSplitString(':', line, list, 5);
+	 char *pkg = list[1];
+	 char *action = _strstrip(list[2]);
+
+	 if(strncmp(action,"error",strlen("error")) == 0)
+	 {
+	    status << "pmerror:" << list[1]
+		   << ":"  << (Done/float(Total)*100.0) 
+		   << ":" << list[3]
+		   << endl;
+	    if(OutStatusFd > 0)
+	       write(OutStatusFd, status.str().c_str(), status.str().size());
+	    line[0]=0;
+	    if (_config->FindB("Debug::pkgDPkgProgressReporting",false) == true)
+	       std::clog << "send: '" << status.str() << "'" << endl;
+	    continue;
+	 }
+	 if(strncmp(action,"conffile",strlen("conffile")) == 0)
+	 {
+	    status << "pmconffile:" << list[1]
+		   << ":"  << (Done/float(Total)*100.0) 
+		   << ":" << list[3]
+		   << endl;
+	    if(OutStatusFd > 0)
+	       write(OutStatusFd, status.str().c_str(), status.str().size());
+	    line[0]=0;
+	    if (_config->FindB("Debug::pkgDPkgProgressReporting",false) == true)
+	       std::clog << "send: '" << status.str() << "'" << endl;
+	    continue;
+	 }
+
+	 vector<struct DpkgState> &states = PackageOps[pkg];
+	 const char *next_action = NULL;
+	 if(PackageOpsDone[pkg] < states.size())
+	    next_action = states[PackageOpsDone[pkg]].state;
+	 // check if the package moved to the next dpkg state
+	 if(next_action && (strcmp(action, next_action) == 0)) 
+	 {
+	    // only read the translation if there is actually a next
+	    // action
+	    const char *translation = states[PackageOpsDone[pkg]].str;
+	    char s[200];
+	    snprintf(s, sizeof(s), translation, pkg);
+
+	    // we moved from one dpkg state to a new one, report that
+	    PackageOpsDone[pkg]++;
+	    Done++;
+	    // build the status str
+	    status << "pmstatus:" << pkg 
+		   << ":"  << (Done/float(Total)*100.0) 
+		   << ":" << s
+		   << endl;
+	    if(OutStatusFd > 0)
+	       write(OutStatusFd, status.str().c_str(), status.str().size());
+	    if (_config->FindB("Debug::pkgDPkgProgressReporting",false) == true)
+	       std::clog << "send: '" << status.str() << "'" << endl;
+
+	 }
+	 if (_config->FindB("Debug::pkgDPkgProgressReporting",false) == true) 
+	    std::clog << "(parsed from dpkg) pkg: " << pkg 
+		      << " action: " << action << endl;
 
-         // Restore sig int/quit
-         signal(SIGQUIT,old_SIGQUIT);
-         signal(SIGINT,old_SIGINT);
-	 return _error->Errno("waitpid","Couldn't wait for subprocess");
+	 // reset the line buffer
+	 line[0]=0;
       }
+      close(_dpkgin);
 
       // Restore sig int/quit
       signal(SIGQUIT,old_SIGQUIT);
diff --git a/apt-pkg/deb/dpkgpm.h b/apt-pkg/deb/dpkgpm.h
index b59b9dc93..2ff8a9ac7 100644
--- a/apt-pkg/deb/dpkgpm.h
+++ b/apt-pkg/deb/dpkgpm.h
@@ -23,6 +23,13 @@ using std::vector;
 class pkgDPkgPM : public pkgPackageManager
 {
    protected:
+
+   // used for progress reporting
+   struct DpkgState 
+   {
+      const char *state;     // the dpkg state (e.g. "unpack")
+      const char *str;       // the human readable translation of the state
+   };
    
    struct Item
    {
@@ -45,7 +52,7 @@ class pkgDPkgPM : public pkgPackageManager
    virtual bool Install(PkgIterator Pkg,string File);
    virtual bool Configure(PkgIterator Pkg);
    virtual bool Remove(PkgIterator Pkg,bool Purge = false);
-   virtual bool Go(int status_fd=-1);
+   virtual bool Go(int StatusFd=-1);
    virtual void Reset();
    
    public:
diff --git a/apt-pkg/init.h b/apt-pkg/init.h
index 74ac3a7ca..e21351797 100644
--- a/apt-pkg/init.h
+++ b/apt-pkg/init.h
@@ -18,7 +18,7 @@
 
 // See the makefile
 #define APT_PKG_MAJOR 3
-#define APT_PKG_MINOR 5
+#define APT_PKG_MINOR 10
 #define APT_PKG_RELEASE 0
     
 extern const char *pkgVersion;
diff --git a/apt-pkg/makefile b/apt-pkg/makefile
index 5f48f0f52..8de7d945e 100644
--- a/apt-pkg/makefile
+++ b/apt-pkg/makefile
@@ -13,7 +13,7 @@ include ../buildlib/defaults.mak
 # methods/makefile - FIXME
 LIBRARY=apt-pkg
 LIBEXT=$(GLIBC_VER)$(LIBSTDCPP_VER)
-MAJOR=3.9
+MAJOR=3.10
 MINOR=0
 SLIBS=$(PTHREADLIB) $(INTLLIBS)
 APT_DOMAIN:=libapt-pkg$(MAJOR)
diff --git a/apt-pkg/packagemanager.cc b/apt-pkg/packagemanager.cc
index a08ccd602..155408bb4 100644
--- a/apt-pkg/packagemanager.cc
+++ b/apt-pkg/packagemanager.cc
@@ -631,11 +631,11 @@ pkgPackageManager::OrderResult pkgPackageManager::OrderInstall()
 // ---------------------------------------------------------------------
 /* This uses the filenames in FileNames and the information in the
    DepCache to perform the installation of packages.*/
-pkgPackageManager::OrderResult pkgPackageManager::DoInstall()
+pkgPackageManager::OrderResult pkgPackageManager::DoInstall(int status_fd)
 {
    OrderResult Res = OrderInstall();
    if (Res != Failed)
-      if (Go() == false)
+      if (Go(status_fd) == false)
 	 return Failed;
    return Res;
 }
diff --git a/apt-pkg/packagemanager.h b/apt-pkg/packagemanager.h
index 43f2c4ace..f64637d03 100644
--- a/apt-pkg/packagemanager.h
+++ b/apt-pkg/packagemanager.h
@@ -76,7 +76,7 @@ class pkgPackageManager : protected pkgCache::Namespace
    // Main action members
    bool GetArchives(pkgAcquire *Owner,pkgSourceList *Sources,
 		    pkgRecords *Recs);
-   OrderResult DoInstall();
+   OrderResult DoInstall(int statusFd=-1); 
    bool FixMissing();
    
    pkgPackageManager(pkgDepCache *Cache);
diff --git a/cmdline/apt-get.cc b/cmdline/apt-get.cc
index 75617d194..e673e0f5b 100644
--- a/cmdline/apt-get.cc
+++ b/cmdline/apt-get.cc
@@ -765,7 +765,8 @@ bool InstallPackages(CacheFile &Cache,bool ShwKept,bool Ask = true,
    if (_config->FindB("APT::Get::Simulate") == true)
    {
       pkgSimulate PM(Cache);
-      pkgPackageManager::OrderResult Res = PM.DoInstall();
+      int status_fd = _config->FindI("APT::Status-Fd",-1);
+      pkgPackageManager::OrderResult Res = PM.DoInstall(status_fd);
       if (Res == pkgPackageManager::Failed)
 	 return false;
       if (Res != pkgPackageManager::Completed)
@@ -994,7 +995,8 @@ bool InstallPackages(CacheFile &Cache,bool ShwKept,bool Ask = true,
       }
        	 
       _system->UnLock();
-      pkgPackageManager::OrderResult Res = PM->DoInstall();
+      int status_fd = _config->FindI("APT::Status-Fd",-1);
+      pkgPackageManager::OrderResult Res = PM->DoInstall(status_fd);
       if (Res == pkgPackageManager::Failed || _error->PendingError() == true)
 	 return false;
       if (Res == pkgPackageManager::Completed)
diff --git a/configure.in b/configure.in
index 7e3aba331..1ee7e168b 100644
--- a/configure.in
+++ b/configure.in
@@ -18,7 +18,7 @@ AC_CONFIG_AUX_DIR(buildlib)
 AC_CONFIG_HEADER(include/config.h:buildlib/config.h.in include/apti18n.h:buildlib/apti18n.h.in)
 
 dnl -- SET THIS TO THE RELEASE VERSION --
-AC_DEFINE_UNQUOTED(VERSION,"0.6.40")
+AC_DEFINE_UNQUOTED(VERSION,"0.6.40.2")
 PACKAGE="apt"
 AC_DEFINE_UNQUOTED(PACKAGE,"$PACKAGE")
 AC_SUBST(PACKAGE)
diff --git a/debian/apt-doc.docs b/debian/apt-doc.docs
new file mode 100644
index 000000000..a7507f4e7
--- /dev/null
+++ b/debian/apt-doc.docs
@@ -0,0 +1 @@
+README.progress-reporting
\ No newline at end of file
diff --git a/debian/changelog b/debian/changelog
index 78b400954..614041acf 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,20 @@
+apt (0.6.40.2) unstable; urgency=low
+
+  * improved the support for "error" and "conffile" reporting from
+    dpkg, added the format to README.progress-reporting
+  * added README.progress-reporting to the apt-doc package
+
+ --
+
+apt (0.6.40.1) unstable; urgency=low
+
+  * bugfix in the parsing code for the apt<->dpkg communication. apt 
+    crashed when dpkg sends the same state more than once under certain
+    conditions
+  * 0.6.40 breaks the ABI but I accidentally didn't change the soname :/
+
+ -- Michael Vogt <mvo@debian.org>  Fri,  5 Aug 2005 13:24:58 +0200
+
 apt (0.6.40) unstable; urgency=low
 
   * Patch from Jordi Mallach to mark some additional strings for translation
@@ -5,8 +22,13 @@ apt (0.6.40) unstable; urgency=low
   * Merge from bubulle@debian.org--2005/apt--main--0:
     - Update pot and merge with *.po
     - Updated French translation, including apt-key.fr.8
+  * Restore changelog entries from the 0.6.x series that went to Debian
+    experimental
+  * Merge michael.vogt@ubuntu.com--2005/apt--progress-reporting--0
+    - Provide an interface for progress reporting which can be used by
+      (e.g.) base-config
 
- -- Matt Zimmerman <mdz@ubuntu.com>  Thu, 28 Jul 2005 11:57:32 -0700
+ -- Matt Zimmerman <mdz@debian.org>  Thu, 28 Jul 2005 11:57:32 -0700
 
 apt (0.6.39) unstable; urgency=low
 
@@ -213,6 +235,236 @@ apt (0.6.27) hoary; urgency=low
 
  -- Matt Zimmerman <mdz@canonical.com>  Mon, 13 Dec 2004 01:03:11 -0800
 
+apt (0.6.25) experimental; urgency=low
+
+  * Fix handling of two-part sources for sources.list deb-src entries in
+    the same way that deb entries were fixed
+
+ -- Matt Zimmerman <mdz@debian.org>  Wed,  9 Jun 2004 05:29:50 -0700
+
+apt (0.6.24) experimental; urgency=low
+
+  * YnPrompt fixes were inadvertently left out, include them (Closes:
+    #249251)
+
+ -- Matt Zimmerman <mdz@debian.org>  Sun, 16 May 2004 14:18:53 -0700
+
+apt (0.6.23) experimental; urgency=low
+
+  * Remove obsolete pkgIterator::TargetVer() (Closes: #230159)
+  * Reverse test in CheckAuth to match new prompt (Closes: #248211)
+
+ -- Matt Zimmerman <mdz@debian.org>  Sun,  9 May 2004 21:01:58 -0700
+
+apt (0.6.22) experimental; urgency=low
+
+  * Merge 0.5.25
+  * Make the unauthenticated packages prompt more intuitive (yes to
+    continue, default no), but require --force-yes in addition to
+    --assume-yes in order to override
+
+ -- Matt Zimmerman <mdz@debian.org>  Fri, 19 Mar 2004 13:55:35 -0800
+
+apt (0.6.21) experimental; urgency=low
+
+  * Merge 0.5.24
+
+ -- Matt Zimmerman <mdz@debian.org>  Tue, 16 Mar 2004 22:52:34 -0800
+
+apt (0.6.20) experimental; urgency=low
+
+  * Merge 0.5.23
+
+ -- Matt Zimmerman <mdz@debian.org>  Thu, 26 Feb 2004 17:17:02 -0800
+
+apt (0.6.19) experimental; urgency=low
+
+  * Merge 0.5.22
+  * Convert apt-key(8) to docbook XML
+
+ -- Matt Zimmerman <mdz@debian.org>  Mon,  9 Feb 2004 15:44:49 -0800
+
+apt (0.6.18) experimental; urgency=low
+
+  * Add new Debian Archive Automatic Signing Key to the default keyring
+    (existing keyrings are not updated; do that yourself)
+
+ -- Matt Zimmerman <mdz@debian.org>  Sat, 17 Jan 2004 17:04:30 -0800
+
+apt (0.6.17) experimental; urgency=low
+
+  * Merge 0.5.21
+  * Handle more IMS stuff correctly
+
+ -- Matt Zimmerman <mdz@debian.org>  Fri, 16 Jan 2004 10:54:25 -0800
+
+apt (0.6.16) experimental; urgency=low
+
+  * Fix some cases where the .gpg file could be left in place when it is
+    invalid
+
+ -- Matt Zimmerman <mdz@debian.org>  Fri,  9 Jan 2004 09:22:15 -0800
+
+apt (0.6.15) experimental; urgency=low
+
+  * s/Debug::Acquire::gpg/&v/
+  * Honor the [vendor] syntax in sources.list again (though it is not
+    presently used for anything)
+  * Don't ship vendors.list(5) since it isn't used yet
+  * Revert change from 0.6.10; it was right in the first place and the
+    problem was apparently something else.  Archive = Suite.
+
+ -- Matt Zimmerman <mdz@debian.org>  Mon,  5 Jan 2004 17:43:01 -0800
+
+apt (0.6.14) experimental; urgency=low
+
+  * Merge 0.5.20
+
+ -- Matt Zimmerman <mdz@debian.org>  Sun,  4 Jan 2004 11:09:21 -0800
+
+apt (0.6.13) experimental; urgency=low
+
+  * Merge 0.5.19
+
+ -- Matt Zimmerman <mdz@debian.org>  Sat,  3 Jan 2004 16:22:31 -0800
+
+apt (0.6.12) experimental; urgency=low
+
+  * Have pkgAcquireIndex calculate an MD5 sum if one is not provided by
+    the method (as with file: and copy:).  Local repositories
+  * Fix warning about dist name mismatch to actually print what it was
+    expecting
+  * Don't expect any particular distribution name for two-part
+    sources.list entries
+  * Merge 0.5.18
+
+ -- Matt Zimmerman <mdz@debian.org>  Fri,  2 Jan 2004 13:59:00 -0800
+
+apt (0.6.11) experimental; urgency=low
+
+  * Support IMS requests of Release.gpg and Release
+  * This required API changes, bump the libapt-pkg version
+  * Copy local Release files into Dir::State::Lists
+  * Set IndexFile attribute when retrieving Release and Release.gpg so
+    that the appropriate Cache-Control headers are sent
+
+ -- Matt Zimmerman <mdz@debian.org>  Fri,  2 Jan 2004 10:46:17 -0800
+
+apt (0.6.10) experimental; urgency=low
+
+  * Use "Codename" (woody, sarge, etc.) to supply the value of the
+    "Archive" package file attribute, used to match "release a=" type
+    pins, rather than "Suite" (stable, testing, etc.)
+
+ -- Matt Zimmerman <mdz@debian.org>  Thu,  1 Jan 2004 16:56:47 -0800
+
+apt (0.6.9) experimental; urgency=low
+
+  * Another tagfile workaround
+
+ -- Matt Zimmerman <mdz@debian.org>  Thu,  1 Jan 2004 13:56:08 -0800
+
+apt (0.6.8) experimental; urgency=low
+
+  * Add a config option and corresponding command line option
+    (--allow-unauthenticated) to apt-get, to make buildd operators happy
+    (Closes: #225648)
+
+ -- Matt Zimmerman <mdz@debian.org>  Wed, 31 Dec 2003 08:28:04 -0800
+
+apt (0.6.7) experimental; urgency=low
+
+  * Forgot to revert part of the changes to tagfile in 0.6.4.  Hopefully
+    will fix segfaults for some folks.
+
+ -- Matt Zimmerman <mdz@debian.org>  Wed, 31 Dec 2003 08:01:28 -0800
+
+apt (0.6.6) experimental; urgency=low
+
+  * Restore the ugly hack I removed from indexRecords::Load which set the
+    pkgTagFile buffer size to (file size)+256.  This is concealing a bug,
+    but I can't fix it right now.  This should fix the segfaults that
+    folks are seeing with 0.6.[45].
+
+ -- Matt Zimmerman <mdz@debian.org>  Mon, 29 Dec 2003 18:11:13 -0800
+
+apt (0.6.5) experimental; urgency=low
+
+  * Move the authentication check into a separate function in apt-get
+  * Fix display of unauthenticated packages when they are in the cache
+    (Closes: #225336)
+
+ -- Matt Zimmerman <mdz@debian.org>  Sun, 28 Dec 2003 16:47:57 -0800
+
+apt (0.6.4) experimental; urgency=low
+
+  * Use the top-level Release file in LoadReleaseInfo, rather than looking
+    for the per-section ones (which aren't downloaded anymore).  This
+    unbreaks release pinning, including the NotAutomatic bit used by
+    project/experimental
+  * Use FileFd::Size() rather than a separate stat() call in
+    LoadReleaseInfo
+  * Fix pkgTagFile to leave a little extra room at the end of the buffer
+    to append the record separator if it isn't present
+  * Change LoadReleaseInfo to use "Suite" rather than "Archive", to match
+    the Debian archive's dist-level Release files
+
+ -- Matt Zimmerman <mdz@debian.org>  Sun, 28 Dec 2003 15:55:55 -0800
+
+apt (0.6.3) experimental; urgency=low
+
+  * Fix MetaIndexURI for flat ("foo/") sources
+
+ -- Matt Zimmerman <mdz@debian.org>  Sun, 28 Dec 2003 12:11:56 -0800
+
+apt (0.6.2) experimental; urgency=low
+
+  * Add space between package names when multiple unauthenticated packages
+    are being installed (Closes: #225212)
+  * Provide apt-key with a secret keyring and a trustdb, even though we
+    would never use them, because it blows up if it doesn't have them
+  * Fix typo in apt-key(8) (standard input is '-', not '/')
+
+ -- Matt Zimmerman <mdz@debian.org>  Sat, 27 Dec 2003 13:01:40 -0800
+
+apt (0.6.1) experimental; urgency=low
+
+  * Merge apt 0.5.17
+  * Rearrange Release file authentication code to be more clear
+  * If Release is present, but Release.gpg is not, don't forget to still
+    queue Packages files
+  * Convert distribution "../project/experimental" to "experimental" for
+    comparison purposes
+  * Make a number of Release file errors into warnings; for now, it is OK
+    not to have a codename, for example.  We mostly care about checksums
+    for now
+
+ -- Matt Zimmerman <mdz@debian.org>  Fri, 26 Dec 2003 15:12:47 -0800
+
+apt (0.6.0) experimental; urgency=low
+
+  * Signature verification support patch ("apt-secure") from Colin Walters
+    <walters@debian.org> and Isaac Jones <ijones@syntaxpolice.org>.  This
+    implements:
+     - Release signature verification (Release.gpg)
+     - Packages, Sources md5sum verification against Release
+     - Closes: #203741
+  * Make some modifications to signature verification support:
+    - Release.gpg is always retrieved and verified if present, rather than
+      requiring that sources be configured as secure
+    - Print a hint about installing gnupg if exec(gpgv) fails
+    - Remove obsolete pkgAcqIndexRel
+    - Move vendors.list stuff into a separate module (vendorlist.{h,cc})
+    - If any files about to be retrieved are not authenticated, issue a
+      warning to the user and require confirmation
+    - Fix a heap corruption bug in pkgSrcRecords::pkgSrcRecords()
+  * Suggests: gnupg
+  * Install a keyring in /usr/share/apt/debian-archive.gpg containing an
+    initial set of Debian archive signing keys to seed /etc/apt/trusted.gpg
+  * Add a new tool, apt-key(8) used to manage the keyring
+
+ -- Matt Zimmerman <mdz@debian.org>  Fri, 26 Dec 2003 08:27:19 -0800
+
 apt (0.5.32) hoary; urgency=low
 
   * Call setlocale in the methods, so that the messages are properly
diff --git a/doc/examples/configure-index b/doc/examples/configure-index
index a09ea0c37..dee0c06ff 100644
--- a/doc/examples/configure-index
+++ b/doc/examples/configure-index
@@ -84,6 +84,13 @@ APT
   Force-LoopBreak "false";         // DO NOT turn this on, see the man page
   Cache-Limit "4194304";
   Default-Release "";
+
+
+  // Write progress messages on this fd (for stuff like base-config)
+  Status-Fd "-1";
+  // Keep the list of FDs open (normally apt closes all fds when it
+  // does a ExecFork)
+  Keep-Fds {};
 };
 
 // Options for the downloading routines
@@ -234,6 +241,7 @@ Debug
   pkgAcquire "false";
   pkgAcquire::Worker "false";
   pkgDPkgPM "false";
+  pkgDPkgProgressReporting "false";
   pkgOrderList "false";
   
   pkgInitialize "false";   // This one will dump the configuration space
diff --git a/methods/makefile b/methods/makefile
index 089300570..06fd2a6fc 100644
--- a/methods/makefile
+++ b/methods/makefile
@@ -7,7 +7,7 @@ include ../buildlib/defaults.mak
 BIN := $(BIN)/methods
 
 # FIXME..
-LIB_APT_PKG_MAJOR = 3.5
+LIB_APT_PKG_MAJOR = 3.10
 APT_DOMAIN := libapt-pkg$(LIB_APT_PKG_MAJOR)
 
 # The file method
diff --git a/po/apt-all.pot b/po/apt-all.pot
index 53be3fcca..d06c14ce1 100644
--- a/po/apt-all.pot
+++ b/po/apt-all.pot
@@ -7,7 +7,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: PACKAGE VERSION\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2005-07-02 11:19-0700\n"
+"POT-Creation-Date: 2005-06-06 14:00+0200\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language-Team: LANGUAGE <LL@li.org>\n"
diff --git a/test/conf_clear.cc b/test/conf_clear.cc
new file mode 100644
index 000000000..259aa0f39
--- /dev/null
+++ b/test/conf_clear.cc
@@ -0,0 +1,23 @@
+#include <apt-pkg/configuration.h>
+#include <apt-pkg/error.h>
+
+using namespace std;
+
+int main(int argc,const char *argv[])
+{
+   Configuration Cnf;
+
+   cout << "adding elements" << endl;
+   Cnf.Set("APT::Keep-Fds::",28);
+   Cnf.Set("APT::Keep-Fds::",17);
+   Cnf.Set("APT::Keep-Fds::",47);
+   Cnf.Dump();
+
+   cout << "Removing  elements" << endl;
+   Cnf.Clear("APT::Keep-Fds",17);
+   Cnf.Clear("APT::Keep-Fds",28);
+   Cnf.Clear("APT::Keep-Fds",47);
+   Cnf.Dump();
+
+   return 0;
+}
diff --git a/test/makefile b/test/makefile
index 9ad6233c2..a9dbdc34d 100644
--- a/test/makefile
+++ b/test/makefile
@@ -39,6 +39,12 @@ SOURCE = testextract.cc
 include $(PROGRAM_H)
 
 # Program for testing the config file parser
+PROGRAM=conftest_clear
+SLIBS = -lapt-pkg
+SOURCE = conf_clear.cc
+include $(PROGRAM_H)
+
+# Program for testing the config file parser
 PROGRAM=conftest
 SLIBS = -lapt-pkg
 SOURCE = conf.cc