OcIc@sdZddklZddkZddkZddkZddkZddkZddkZddk Z ddk Z ddk Z ddk Z ddk Z ddkZddkZddkZddkZddkZddkZddkZdZdZdddYZe ieddd YZe ied dd YZd eifd YZdefdYZdS(s:High level functions for mirroring and mirror+incrementingi(t generatorsNcCstid|i|ifd|iii}|iii}|i}|i||d|i |}|i |}|i ||dS(s(Turn dest_rpath into a copy of src_rpathsStarting mirror %s to %siiN( tlogtLogtpathtconntbackupt SourceStructtDestinationStructtget_source_selecttset_rorp_cachetget_sigst get_diffstpatch(t src_rpatht dest_rpathtSourceStDestSt source_rpitert dest_sigitertsource_diffiter((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytMirrors  cCstid|i|ifd|iii}|iii}|i}|i||d|i |}|i |}|i |||dS(s2Mirror + put increments in tree based at inc_rpaths%Starting increment operation %s to %siiN( RRRRRRRRR R R tpatch_and_increment(R Rt inc_rpathRRRRR((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytMirror_and_increment(s  RcBs/eZdZdZdZdZdZRS(s-Hold info used on source side when backing upcGs_ti|}|i|||itid}ti|||_ti d|dS(sjInitialize select object using tuplelist Note that each list in filelists must each be passed as separate arguments, so each is recognized as a file by the connection. Otherwise we will get an error because a list containing files can't be pickled. Also, cls._source_select needs to be cached so get_diffs below can retrieve the necessary rps. it select_mirrorN( t selectiontSelectt ParseArgstset_itertGlobalstpipeline_max_lengthtrorpitertCacheIndexablet_source_selecttset(tclstrpatht tuplelistt fileliststselt cache_size((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytset_source_select9s   cCs|iS(s7Return source select iterator, set by set_source_select(R!(R#((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRLsc#s|i}tidfd}fd}x|D]}|tijotiVq=n|i|ipti |i}|i }|i o|i |i nW|io2|io||||q |||n|i|id|Vq=WdS(s8Return diffs of any files with signature in dest_sigitert ListErrorcs]titii|df}|o|iti|n |i|i ddS(s7Attach file of snapshot to diff_rorp, w/ error checkingtrbtsnapshotN( trobusttcheck_common_errorR$tRPathtopentsetfilethasht FileWrappertzerotset_attached_filetype(t diff_rorptsrc_rptfileobj(t error_handler(s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytattach_snapshotTs  cs^titi||f}|o|i||idn|i|iddS(s3Attach file of diff to diff_rorp, w/ error checkingtdiffR,N(R-R.tRdifftget_delta_sigrp_hashR1R5R4(R6R7tdest_sigR8(R9(s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt attach_diff\s  R,N(R!R-tget_error_handlertiterfiletMiscIterFlushRepeatt MiscIterFlushtgettindexR$tRORPatht getRORPatht isflaglinkedt flaglinkedt get_link_flagtisregtclose_if_necessaryR5(R#Rt source_rpsR:R?R>R7R6((R9s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR Ps*        N(t__name__t __module__t__doc__tNoneR!R)RR (((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR6s   RcBsSeZdZddZdZdZdZdZd dZdZ RS( s2Hold info used by destination side when backing upicsOfd}ti|o(tiiti}|o|SqHn|S(sReturn destination select rorpath iterator If metadata file doesn't exist, select all files on destination except rdiff-backup-data directory. cs#ti}|i|iS(s-Get the combined iterator from the filesystem(RRtparse_rbdir_excludeR(R'(R$(s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytget_iter_from_fss (tmetadatat SetManagert ManagerObjt GetAtTimetTimetprevtime(R#R$t use_metadataRSt rorp_iter((R$s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytget_dest_select}s  cCsD|i||}ti||}t|tid||_dS(sInitialize cls.CCPP, the destination rorp cache for_increment should be true if we are mirror+incrementing, false if we are just mirroring. iN(R\Rt Collate2IterstCacheCollatedPostProcessRRtCCPP(R#tbaserpt source_itert for_incrementt dest_itertcollated((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR sccstid}d}x|iD]\}}titij o-|d7}||jod}tiVqin|o/|o(||joti pti ||pV|o |i p|i }|i ||||}|o|ii ||VqqqWdS(sYield signatures of any changed destination files If we are backing up across a pipe, we must flush the pipeline every so often so it doesn't get congested on destination end. iiiN( RRR_t backup_readert backup_writerRARBtpreserve_hardlinkstHardlinktrorp_eqREt get_one_sigt flag_changed(R#tdest_base_rpathtflush_thresholdtnum_rorps_seentsrc_rorpt dest_rorpREtsig((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR s$       cCstio@|o9ti|o)ti|}|iti|n{|od|i}|i oGt i ||}|i |}|djodSn|i|qnti|}|S(s5Return a signature given source and destination rorpsN(RRgRhtislinkedR$RFRItget_link_indexRGRKtlongnamet get_mirror_rptget_one_sig_fpRQR1(R#RlRERoRpR>tdest_rptsig_fp((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRjs   cCs|iptiid|ddSntidjo6|i o(|io|i d|i Bnyt i |SWnt j o}|itijoby(|i d|i Bt i |SWq t tfj o!tiid|ifq XqnXdS(s?Return a signature fp of given index, corresponding to reg filet UpdateErrors/File changed from regular file before signatureiis9Could not open %s for reading. Check permissions on file.N(RKRtErrorLogt write_if_openRQRt process_uidtreadabletisownertchmodtgetpermsR<t get_signaturetIOErrorterrnotEPERMtOSErrorRt FatalErrorR(R#Rwte((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRvs$   cCstit||ig}xDti||D]0}tid|id||i|q.W|i |ii |i dS(s*Patch dest_rpath with an rorpiter of diffssProcessing changed file iN( RtIterTreeReducert PatchITRBR_t FillInIterRRt get_indexpathREtFinishtclosetsetdata(R#RRt start_indextITRR;((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR s  cCstit|||ig}xDti||D]0}tid|id||i|q1W|i |ii |i dS(s<Patch dest_rpath with rorpiter of diffs and write incrementssProcessing changed file iN( RRt IncrementITRBR_RRRRRERRR(R#RRRRR;((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs   (( RNRORPR\R R RjRvR R(((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR{s     R^cBseZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZRS(sCache a collated iter of (source_rorp, dest_rorp) pairs This is necessary for three reasons: 1. The patch function may need the original source_rorp or dest_rp information, which is not present in the diff it receives. 2. The metadata must match what is stored in the destination directory. If there is an error, either we do not update the dest directory for that file and the old metadata is used, or the file is deleted on the other end.. Thus we cannot write any metadata until we know the file has been procesed correctly. 3. We may lack permissions on certain destination directories. The permissions of these directories need to be relaxed before we enter them to computer signatures, and then reset after we are done patching everything inside them. 4. We need some place to put hashes (like SHA1) after computing them and before writing them to the metadata. The class caches older source_rorps and dest_rps so the patch function can retrieve them if necessary. The patch function can also update the processed correctly flag. When an item falls out of the cache, we assume it has been processed, and write the metadata for it. cCs||_||_||_ti|_tiotii nt i i |_ h|_g|_g|_g|_dS(sInitialize new CCWP.N(titerR(t dest_root_rpt statisticstinit_statfileobjt statfileobjRtfile_statisticst FileStatstinitRTRVt GetWritert metawritert cache_dicttcache_indiciestdir_perms_listt parent_list(tselft collated_iterR(R((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt__init__s       cCs|S(N((R((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt__iter__<scCs|ii\}}|i|||o |ip|i}||dddg|i|<|ii|t|i|i jo|i n||fS(s@Return next (source_rorp, dest_rorp) pair. StopIteration passediN( Rtnextt pre_processRERQRRtappendtlenR(t shorten_cache(Rt source_rorpRpRE((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR>scCsxtio|oti||n|oH|io;tidjo+|iddjo|i||ndS(sDo initial processing on source_rorp and dest_rorp It will not be clear whether source_rorp and dest_rorp have errors at this point, so don't do anything which assumes they will be backed up correctly. iiiN(RRgRhtadd_rorptisdirR|Rtunreadable_dir_init(RRRp((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRIs $cCsd|ii|i}|id|iB|o-|io |ii||ifndS(sInitialize an unreadable dir. Make it readable, and if necessary, store the old permissions in self.dir_perms_list so the old perms can be restored. iN(Rt new_indexRERRRRR(RRRpRw((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRWsc Cs|id}|id=y |i|\}}}}}Wn.tj o"tid|fddSnX|i|=|i||||||io|i|n|i|||dS(s=Remove one element from cache, possibly adding it to metadatais(Warning index %s missing from CCPP cacheiN( RRtKeyErrorRRt post_processRtreset_dir_permstupdate_parent_list(Rt first_indextold_source_rorpt old_dest_rorpt changed_flagt success_flagtinc((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRcs         cCs|o |ip|o |ipdSn|ip|djp t|n|idd}t|t|}}||jo%||djpt||fn4||d |d jp t||i| |_|ii|||ffdS(sIUpdate the parent cache with the recently expired main cache entry This method keeps parent directories in the secondary parent cache until all their children have expired from the main cache. This is necessary because we may realize we need a parent directory's information after we have processed many subfiles. Niii((RRtAssertionErrorRR(RRERoRptlast_parent_indextlp_indextli((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRss   %#cCs#tio|oti|n| p|o:|o|ii|n|o|ii|qkn|djo |}n|djo |}nd}|djp |djo|ii||n|o!|i o|i i |nti ot ii||||ndS(s Post process source_rorp and dest_rorp. The point of this is to write statistics and metadata. changed will be true if the files have changed. success will be true if the files have been successfully updated (this is always false for un-changed files). iiiN(RRgRhtdel_rorpRtadd_source_filet add_dest_fileRQt add_changedtlstatRt write_objectRRRtupdate(RRRptchangedtsuccessRt metadata_rorp((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs$    cCsU|id\}}|i}||jo(|t| |jo|i|ndS(s;Reset the permissions of directories when we have left themiN(RRERR(Rt current_indextdir_rptpermst dir_index((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs   cCs|ii|S(s$Return true if given index is cached(Rthas_key(RRE((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytin_cachescCsd|i|dSignal that the file with given index was updated successfullyiiN(R(RRE((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt flag_successscCsd|i|dy|i|dSWn$tj o|i|dSnXdS(s0Retrieve mirror_rorp with given index from cacheiN(RRR(RRE((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytget_mirror_rorpscCs|i|i|dS(s"Update the source rorp's SHA1 hashN(Rtset_sha1(RREtsha1sum((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt update_hashscCsAti|}|pdSn|i|i}|i|dS(s=Tag associated source_rorp with same hash diff_rorp points toN(Rhtget_sha1RRER(RR6RR((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytupdate_hardlink_hashs cCsx|io|iqWx1|io&|ii\}}|i|qW|iitii t i ot i nt iot iint idS(s+Process the remaining elements in the cacheN(RRRtpopRRRRTRVtConvertMetaToDiffRtprint_statisticsRtprint_active_statsRRtwrite_active_statfileobj(RRR((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs      (RNRORPRRRRRRRRRRRRRkRRRRRRRR(((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR^s, "                 RcBseZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZRS(sMPatch an rpath with the given diff iters (use with IterTreeReducer) The main complication here involves directories. We have to finish processing the directory after what's in the directory, as the directory may have inappropriate permissions to alter the contents or the dir's mtime could change as we change the contents. cCsp||_|itijpttip ti|_d\|_ |_ ||_ t id|_dS(s9Set basis_root_rp, the base of the tree to be incrementedRyN(NN(t basis_root_rpRRtlocal_connectionRRtget_active_statfileobjt StatFileObjRRQtdir_replacementt dir_updateR_R-R@R9(RRR_((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs   cCs1|ii|}|ip|o |i S(s0True if diff_rorp and mirror are not directories(R_RR(RRER6t mirror_rorp((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pytcan_fast_processscCsti|ii||i\}}|i p t|ti|}|i |||o|i oJt i |i ti||fdjo|ii|q|iq|o+|i o|i|ii|qn&|i|i o|indS(s>Patch base_rp with diff_rorp (case where neither is directory)N(Rttget_mirror_inc_rpsR_RRRRtTempFiletnewt patch_to_tempRR-R.R9R$trenameRQRtdeleteRR(RRER6t mirror_rptdiscardttf((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt fast_process s !    cCs|io|i||nl|idjo:|i||}|pdSq|djodSqn|i|||pdSn|io"|i oti||n|i||S(sPatch basis_rp, writing output in new, which doesn't exist yet Returns true if able to write new as desired, false if UpdateError or similar gets in the way. R,iii( RHtpatch_hardlink_to_temptget_attached_filetypetpatch_snapshot_to_temptpatch_diff_to_tempRR$t copy_attribstmatches_cached_rorp(Rtbasis_rpR6Rtresult((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs  cCs*ti|||i|ii|dS(s4Hardlink diff_rorp to temp, update hash if necessaryN(Rhtlink_rpRR_R(RR6R((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR2scCs|io(|i||ti||dSnti|iti||f}t|t i o!|i i |i |idSn|djS(sWrite diff_rorp to new, return true if successful Returns 1 if normal success, 2 if special file is written, whether or not it is successful. This is because special files either fail with a SpecialFileError, or don't need to be compared. iii(t isspecialt write_specialR$RR-R.R9tcopyt isinstanceR2tReportR_RREt sha1_digest(RR6Rtreport((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR7s cCs||idjptti|iti|||f}t|ti o!|i i |i |i dSn|djS(s0Apply diff_rorp to basis_rp, write output in newR;ii(RRR-R.R9R<t patch_localRR2RR_RRER(RRR6RR((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRLs cCsm|ipdSn|ii|i}|o|i|odSntiid|d|ifdS(sReturn true if new_rp matches cached src rorp This is a final check to make sure the temp file just written matches the stats which we got earlier. If it doesn't it could confuse the regress operation. This is only necessary for regular files. iRys1Updated mirror temp file %s does not match sourcei( RKR_RREt equal_looseRRzR{R(RR6tnew_rpt cached_rorp((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRVs cCsktid}ti|ti||fdjo3|i|io|in|indS(s1Write diff_rorp (which holds special file) to newtSpecialFileErroriN( R-R@R.R$RRRRttouch(RR6Rteh((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRfs %  cCsti|ii||i\|_}|io|i||inL|i||io5|i o|ii |q|ii |ndS(s9Start processing directory - record information for laterN( RtRR_RRtbase_rpRt prepare_dirtset_dir_replacementRRR(RRER6R((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt start_processns$  cCs|idjptti||_|id||ipK|iio|iint i |i i |i |idSndSdS(sSet self.dir_replacement, which holds data until done with dir This is used when base_rp is a dir, and diff_rorp is not. Returns 1 for success or 0 for failure R,iiN(RRRRRRRQRRR$tcopy_with_attribsR_RRE(RR6R ((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR ws cCs|i|_|ipI|io|iin|i|i|ii |i n.|ii |i o|ii |i ndS(s!Prepare base_rp to be a directoryN( RGRRRR RRtmkdirR_RRER(RR6R ((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR s    cCs|io||iiptti|i|itidjo;|iiddjo!|ii d|iiBqnF|i o;|ii |i i oti |i |iqndS(sFinish processing directoryiiiN(RR RRR$RRR|RRRtrmdirRR(R((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyt end_processs %  (RNRORPRRRRRRRRRR R R R(((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs          RcBs)eZdZdZdZdZRS(swPatch an rpath with the given diff iters and write increments Like PatchITRB, but this time also write increments. cCs ||_ti|||dS(N(t inc_root_rpRR(RRRt rorp_cache((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs cCsyti|ii||i|i\}}ti|}|i|||ot i |i t i |||f}|dj ot|t o|ii|||io|in|ioJt i |i ti||fdjo|ii|qD|in,|io|i|ii|ndSqPn|i|io|indS(sAPatch base_rp with diff_rorp and write increment (neither is dir)N(RtRR_RRRRRRR-R.R9t incrementt IncrementRQRtintRRKtfsync_with_dirRR$RRRRR(RRER6Rt inc_prefixRR((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs,'         cCsUti|ii||i|i\|_}|ii|ip9|iip)t dt |i t |ii f|io^t i ||i|}|o|io|in|ii|i||ind|i||ioMt i |i|i|}|o'|ii|||ii|qQndS(sStart processing directorys#Either %s or %s must be a directoryN(RtRR_RRRR RRRRRRRRKRR R RRR(RRER6RR((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyR s$*     (RNRORPRRR (((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyRs  ((((RPt __future__RRRRTRRRhR-RR$tstaticRRRXR<RRAR2RtRRRt MakeClassRR^t ITRBranchRR(((s8/var/lib/python-support/python2.5/rdiff_backup/backup.pyss  B }