Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to Add Multiple Files and Folders to an Archive Using a Single Command #136

Closed
byplt opened this issue Oct 11, 2024 · 46 comments
Closed

Comments

@byplt
Copy link

byplt commented Oct 11, 2024

Similar to 7-Zip;
7z a archive.7z @files.txt

files.txt:

D:\app.exe
D:\testfolder\
@fcorbelli
Copy link
Owner

-input

@byplt
Copy link
Author

byplt commented Oct 11, 2024

thanks

@byplt byplt closed this as completed Oct 11, 2024
@byplt
Copy link
Author

byplt commented Oct 11, 2024

Another question:
If I want to add these files to a different folder in the archive, how can I do that?

for example;
zpaqfranz a archive.zpaq -input "files.txt" -to "in_archive_folder/"

And when listing the archive contents:

zpaqfranz l archive.zpaq

Output:

...
in_archive_folder\file1.txt
in_archive_folder\file2.txt
...

@byplt byplt reopened this Oct 11, 2024
@fcorbelli
Copy link
Owner

-find and -replace

@byplt
Copy link
Author

byplt commented Oct 11, 2024

I think there might be a bug:

(working) zpaqfranz.exe a archive.zpaq "D:\files" -find "D:\files" -replace "test"
(not working) zpaqfranz.exe a archive.zpaq -input "D:\filepaths.txt" -find "D:\files" -replace "test"

Nothing happens.

@fcorbelli
Copy link
Owner

...Because you are using Windows

Please try, with the upgrade -force, the current 60.8b.
This should fix

@byplt
Copy link
Author

byplt commented Oct 12, 2024

It's working, thank you for your effort! There's one more thing:
When using -find and -replace with -input, the archive keeps growing.

For example, when running this command multiple times:
zpaqfranz.exe a archive.zpaq -input "D:\filepaths.txt" -find "D:\files" -replace "test"
output will be;

Ekran görüntüsü 2024-10-12 164512

but, I guess, there should be warning, like;

Ekran görüntüsü 2024-10-12 164649

@fcorbelli
Copy link
Owner

-find/replace will start again the archiving
You can use -to, this should be "smarter"

@byplt
Copy link
Author

byplt commented Oct 13, 2024

thx

@byplt byplt closed this as completed Oct 13, 2024
@byplt
Copy link
Author

byplt commented Oct 15, 2024

Hello again,

I want to preserve the existing files in the folder when adding new files using the "-to" switch. How can I do that?

For example, I run these commands in order:
zpaqfranz.exe a "archive.zpaq" "folder/" -to "newfolder/"
and then:
zpaqfranz.exe a "archive.zpaq" "anotherfolder/" -to "newfolder/"

But the other files are getting removed.
image

@byplt byplt reopened this Oct 15, 2024
@fcorbelli
Copy link
Owner

They are not removed
Zpaq does not remove anything (unless the trim command)
You can list and extract with -until X
Or -comment, if you want

@byplt
Copy link
Author

byplt commented Oct 15, 2024

I understand,

It works like folder synchronization. I was asking if there's a way to turn that off. I only want to include conflicting files in the process. This issue happens when adding folders, but not when adding files one by one. Also, using the -until switch makes version control quite difficult for automation.

image

Thank you for your interest and support.

@fcorbelli
Copy link
Owner

I don't quite understand what you want to do.
You can mark up a version, in zpaqfranz, with -comment something
path manipulation is not at all designed for some kind of “weird” archiving. It serves during the file extraction phase to put them in a different folder than they were originally in

zpaqfranz a z:\okane c:\nz -to "/whereveruno" -comment "uno"
zpaqfranz a z:\okane c:\1200 -to "/whereverdue" -comment "due"

zpaqfranz l z:\okane.zpaq -only "/whereverdue*" -comment "due"

@byplt
Copy link
Author

byplt commented Oct 15, 2024

Let me explain it this way;

image

because I always need the latest version of the archive, working with "-until" and "-comment" too complicated for this situation

@fcorbelli
Copy link
Owner

Why do you want the very same folder?
Do not use -to at all

@byplt
Copy link
Author

byplt commented Oct 15, 2024

I have to use the -to switch; otherwise, I would have to move all the files into a single folder named 'folder,' (before of using zpaqfranz), which is not practical.

Why do I need this? I need drag-and-drop functionality, similar to a GUI version of an archiver. So, if I drop files into a folder (inside the archive), I need the -to switch.

@byplt
Copy link
Author

byplt commented Oct 15, 2024

I know zpaq doesn't work with this logic, but it already has the -to capability, even though it only works partially.

@fcorbelli
Copy link
Owner

This is weird but I can add a -nodelete switch
The file list will only grow

@byplt
Copy link
Author

byplt commented Oct 15, 2024

If this causes the growth, I don't know if it is a 'smarter' way.

@mirogeorg
Copy link

Franco, reading this post, I am truly amazed at how you remember so many options. I guess the source code is often your help ;)

But where can we also read more in detail about them? Specifically, the -input option caught my attention. It's not in the Wiki, but that's where the detailed documentation should be, where a person can open it and read everything about option 'a' in depth.

In the command line help, it's mentioned in five words, but that help is too concise for someone to learn from it. It's more for reference, to remind oneself if they already know what they're doing.

The issue with -input -find -replace wasn't clear to me what it's caused by, but it gets resolved with -to. It really seems more intuitive and clear with -find and -replace, but if it doesn’t work... when and why, at least, I didn't understand... that should go to help, with examples, cases and their solution.

The proposed new option -nodelete... it can't be explained in one line. I understood what it's for, but with just one line, it won't be clear ;)

@fcorbelli
Copy link
Owner

If this causes the growth, I don't know if it is a 'smarter' way.

You can try

@fcorbelli
Copy link
Owner

Franco, reading this post, I am truly amazed at how you remember so many options. I guess the source code is often your help ;)

I wrote them, so it is not that difficult. What is difficult are the portions written by Mahoney

But where can we also read more in detail about them? Specifically, the -input option caught my attention. It's not in the Wiki, but that's where the detailed documentation should be, where a person can open it and read everything about option 'a' in depth.

Usually I write a couple of lines in the Release
Don't forget one key point. zpaqfranz I wrote it for my jobs, where I get paid. While I am not, to publish it
Even just writing two lines takes time, and especially when I have no feedback I am not stimulated

In the command line help, it's mentioned in five words, but that help is too concise for someone to learn from it. It's more for reference, to remind oneself if they already know what they're doing.

True

The issue with -input -find -replace wasn't clear to me what it's caused by, but it gets resolved with -to. It really seems more intuitive and clear with -find and -replace, but if it doesn’t work... when and why, at least, I didn't understand... that should go to help, with examples, cases and their solution.

wow, you really want a big explanation of how zpaq file nomenclature works, and zpaqfranz extensions?
It probably takes a couple of hours to write.
In this particular case, the drama is about using Windows-style () bars instead of zpaq-style (/) ones in paths. This is the short version

The proposed new option -nodelete... it can't be explained in one line. I understood what it's for, but with just one line, it won't be clear ;)

I'll write it down in the meantime, then we'll see if it's of any use, in which case I'll write two lines 😄

@fcorbelli
Copy link
Owner

Just a note. Mahoney is certainly a genius at writing compression algorithms, but he knows virtually nothing about backups; it's not his “world.” This causes zpaq to be born with serious flaws, and incredible complications.
Which, however, I did NOT remove, so as not to “break” backward compatibility. Virtually any zpaq command works identically on zpaqfranz (except for the -summary switch, which was so badly written that I changed it out of desperation). So if you ask yourself “why is there no uniform behavior? Why these strange things?” The answer is: backward compatibility.
Or is it a feature, not a bug

@fcorbelli
Copy link
Owner

BTW

	g_programflags.add(&flag715,			"-715",					"Runs just about like zpaq 7.15",					"a;");
	g_programflags.add(&flagappend,			"-append",				"Append-only (antiransomware, slow)",				"a;");
	g_programflags.add(&flagbackupxxh3,		"-backupxxh3",			"Use XXH3 in backup instead of MD5",				"");
	g_programflags.add(&flagbig,			"-big",					"Big",												"all;");
	g_programflags.add(&flagchecksum,		"-checksum",			"Do checksums",										"");
	g_programflags.add(&flagchecktxt,		"-checktxt",			"Checktxt (MD5)",									"");
#ifdef _WIN32
	g_programflags.add(&flagsfx,			"-sfx",					"Enable SFX module",								"");
#endif
	g_programflags.add(&flagfasttxt,		"-fasttxt",				"Create test .txt with CRC-32 and QUICK",											"");
	g_programflags.add(&flagcomment,		"-comment",				"Comment version",									"");
	g_programflags.add(&flagdebug,			"-debug",				"Show debug line number",							"");
	g_programflags.add(&flagdebug2,			"-debug2",				"Show debug infos",									"");
	g_programflags.add(&flagdebug3,			"-debug3",				"Show A LOT of debug",								"");
	g_programflags.add(&flagdebug4,			"-debug4",				"Write debug data on z:\\",							"");
	g_programflags.add(&flagdebug5,			"-debug5",				"Show RAM usage",							"");
	g_programflags.add(&flagdesc,			"-desc",				"Orderby desc",										"");
	g_programflags.add(&flagnosymlink,		"-symlink",				"Ignore Windows REPARSE_POINT (Symlink)",										"");
	g_programflags.add(&flagdonotforcexls,	"-xls",					"Do NOT force adding of XLS/PPT (default: NO)",		"a;");
	g_programflags.add(&flagdummy,			"-debug -zero -kill",	"Add 0-byte long file (debugging)",					"");
	g_programflags.add(&flagdummy,			"-debug -zero",			"Add files but zero-filled (debugging)",			"");
	g_programflags.add(&flagfilelist,		"-filelist",			"Create a filelist .txt",							"");
#ifdef _WIN32
	g_programflags.add(&flagads,			"-ads",					"Store filelists on NTFS",							"");
#endif
	g_programflags.add(&flagfast,			"-fast",				"Store filelists inside",							"");
	g_programflags.add(&flagfix255,			"-fix255",				"Fix 255",											"");
	g_programflags.add(&flagfixeml,			"-fixeml",				"Fix eml filenames",								"");
	g_programflags.add(&flagflat,			"-flat",				"Flat filenames",									"");
	g_programflags.add(&flagforce,			"-force",				"Force",											"");
	g_programflags.add(&flagforcewindows,	"-forcewindows",		"Store ADS stuff                (default: NO)",		"a;");
	g_programflags.add(&flagforcezfs,		"-forcezfs",			"Enforce using .zfs",								"");
	g_programflags.add(&flagfrugal,			"-frugal",				"Frugal backup",									"");
	g_programflags.add(&flaghashdeep,		"-hashdeep",			"Hashdeep",											"");
	g_programflags.add(&flagignore,			"-ignore",				"Ignore (do not show) file errors",												"a;");
	g_programflags.add(&flaghome,			"-home",				"Home",												"a;");
	g_programflags.add(&flagkill,			"-kill",				"Kill",												"");
	g_programflags.add(&flaght,				"-ht",					"Enable Hyperthread (if any)",												"");
	g_programflags.add(&flagnocaptcha,		"-nocaptcha",			"Skip chaptcha",												"");
	g_programflags.add(&flagmm,				"-mm",					"Memory mapped",									"");
	g_programflags.add(&flagnoattributes,	"-noattributes",		"Do not store attribute",									"");
	g_programflags.add(&flagattr,			"-attr",				"Show attribute (listing)",									"");
	g_programflags.add(&flagthunderbird,	"-thunderbird",			"Thunderbird's e-mail",									"");
	g_programflags.add(&flagnodedup,		"-nodedup",				"Turn off deduplicator",							"");
	g_programflags.add(&flagnoeta,			"-noeta",				"Do not show ETA",									"");
	g_programflags.add(&flagnopath,			"-nopath",				"Do not store path",								"");
	g_programflags.add(&flagnoqnap,			"-noqnap",				"Exclude QNAP snap & trash",						"a;");
	g_programflags.add(&flagnomac,			"-nomac",				"Exclude Mac hidden files",						"a;");
	g_programflags.add(&flagnosynology,		"-nosynology",			"Exclude Synology system files",						"a;");
	g_programflags.add(&flagnorecursion,	"-norecursion",			"Do not recurse into folders (default: YES)",		"");
	g_programflags.add(&flagnosort,			"-nosort",				"Do not sort file",									"");
	g_programflags.add(&flagpakka,			"-pakka",				"New output",										"");
	g_programflags.add(&flagdistinct,		"-distinct",			"PAKKA zpaqlist",										"");
	g_programflags.add(&flagparanoid,		"-paranoid",			"Paranoid",											"");
	///g_programflags.add(&flagpaq,			"-paq",					"Enable paq-levels",											"");
	g_programflags.add(&flagcollision,		"-collision",			"Collision check",											"");
	g_programflags.add(&flagramdisk,		"-ramdisk",				"Ramdisk (really: heap)",											"");
	g_programflags.add(&flagrename,			"-rename",				"Rename",											"");
	g_programflags.add(&flagsilent,			"-silent",				"Silent",											"");
	g_programflags.add(&flagnoconsole,		"-noconsole",			"Do not manipulate console",						"",flagnoconsole);
	g_programflags.add(&flagskipzfs,		"-zfs",					"Do NOT ignore .zfs             (default: YES)",	""); //flagskipzfs
	g_programflags.add(&flagspace,			"-space",				"Do not check space/writeability",					"");
	g_programflags.add(&flagssd,			"-ssd",					"SSD",												"");
	g_programflags.add(&flaghdd,			"-hdd",					"HDD",												"");
	g_programflags.add(&flagstat,			"-stat",				"Statistics",										"");
	g_programflags.add(&flagstdin,			"-stdin",				"stdin",											"");
	g_programflags.add(&flagterse,			"-terse",				"Create output without header and footer",			"");
	g_programflags.add(&flagnodel,			"-nodel",				"Do not show deleted files",			"");
	g_programflags.add(&flagstdout,			"-stdout",				"stdout",											"");
	g_programflags.add(&flagstore,			"-store",				"Store mode: no deduplication, no compression",		"");
	g_programflags.add(&flagtar,			"-tar",					"TAR mode (store posix)",							"");
	g_programflags.add(&flagtest,			"-test",				"Only do test",										"");
	g_programflags.add(&flagtouch,			"-touch",				"Force 'touch' on date (7.15 to zpaqfranz)",		"");
	g_programflags.add(&flagutc,			"-utc",					"Use UTC time",										"");
	g_programflags.add(&flagdate,			"-date",				"Show/save creation date (if possible)",			"");
	g_programflags.add(&flagutf,			"-utf",					"UTF-8",											"");
	g_programflags.add(&flagverbose,		"-verbose",				"Verbose output",									"");
	g_programflags.add(&flagverify,			"-verify",				"Verify",											"");
	g_programflags.add(&flagvss,			"-vss",					"Enable Volume Shadow Copies",						"a;");
	g_programflags.add(&flagzero,			"-zero",				"Flag zero",										"");
	g_programflags.add(&flagpause,			"-pause",				"Pause after run (for runhigh)",					"");
	g_programflags.add(&flagquiet,			"-quiet",				"Do not show filesystem errors",												"");

	g_programflags.add(&g_sfxflagforce,		"-sfxforce",			"Sfx force",									"");
	g_programflags.add(&g_sfxflagall,		"-sfxall",				"Sfx all",										"");

	g_programflags.add(&flagbarraod,		"/od",					"Order by date",									"dir;");
	g_programflags.add(&flagbarraon,		"/on",					"Order by name",									"dir;");
	g_programflags.add(&flagbarraos,		"/os",					"Order by size",									"dir;");

	g_programflags.add(&flagnocolor,		"-nocolor",				"Monochrome output",								"",flagnocolor);

#ifdef _WIN32
///	g_programflags.add(&flagdd,				"-dd",					"dd",												"");
	g_programflags.add(&flagfindzpaq,		"-findzpaq",			"Search .zpaq in every drive letter (USB device)",	"");
	g_programflags.add(&flagfixcase,		"-fixcase",				"Fix CAse",											"");
	g_programflags.add(&flagfixreserved,	"-fixreserved",			"fixreserved",										"");
	g_programflags.add(&flagimage,			"-image",				"Drive image",										"");
	g_programflags.add(&flaglongpath,		"-longpath",			"Longpath",											"");
	g_programflags.add(&flagopen,			"-open",				"open",												"");
#endif

	g_programflags.add(&flaghw,				"-hw",					"Use HW SHA1",										"a;x;");
  • more :)

@fcorbelli
Copy link
Owner

Let me explain it this way;

image

because I always need the latest version of the archive, working with "-until" and "-comment" too complicated for this situation

The problem is: what if the same filename is inside TWO folders?
What if in folderB there is a 1.txt too?

@fcorbelli
Copy link
Owner

You can try the pre-release 60.8e via a zpaqfranz -upgrade -force
There is the brand new switch "-nodelete"
BUT

YOU must handle filename collisions

zpaqfranz.exe a z:\1 c:\zpaqfranz\test1 -to "/posizione" -nodelete
zpaqfranz.exe a z:\1 c:\zpaqfranz\test2 -to "/posizione" -nodelete

On

 Directory di c:\zpaqfranz\test1
16/10/2024  11:01             7.859 1.txt
05/10/2024  16:51            31.109 2.txt
           
 Directory di c:\zpaqfranz\test2
02/07/2024  14:59            23.183 3.txt
15/06/2024  19:58           446.761 4.txt

You get

Z:\omare\posizione>zpaqfranz l z:\1.zpaq
(...)
   Date      Time   s Ratio Name
---------- -------- -------- ----- -----
2024-10-16 11:16:27 s   dir # /posizione/
2024-10-16 11:16:29 s >999% # /posizione/1.txt
2024-10-05 16:51:10 s   23% + /posizione/2.txt
2024-07-02 14:59:08 s   20% + /posizione/3.txt
2024-06-15 19:58:24 s   20% + /posizione/4.txt

@fcorbelli
Copy link
Owner

TRANSLATION
you have two folders
test1 with 1.txt, 2.txt
test2 with 1.txt, 3.txt, 4.txt
and you -to "/posizione"
The .zpaq will contain 1.txt from FOLDER2, 2.txt (folder1), 3.txt and 4.txt (folder2) if you archive test1 before test2

You will get something like

70156! *** UPDATED FILE with -nodelete *** <</posizione/1.txt>>

meaning that the file 1.txt is "overwritten" in the 2nd version (you can take it from the first, in this example)

@fcorbelli
Copy link
Owner

OK, 60.8h just uploaded, with yellow warnings and red errors
CUL

@fcorbelli
Copy link
Owner

Franco, reading this post, I am truly amazed at how you remember so many options. I guess the source code is often your help ;)

But where can we also read more in detail about them?

We can make a deal
I write, YOU read, and report not so clear points

https://github.com/fcorbelli/zpaqfranz/wiki/Voodoo-switches

@byplt
Copy link
Author

byplt commented Oct 16, 2024

Thanks!
It seems to be working, and there is no the growth issue. I will check it thoroughly.

@mirogeorg
Copy link

Franco, with pleasure, but where should I write and where should I ask? Here in Issues?

@fcorbelli
Copy link
Owner

I can make the wiki writeable by anyone

@byplt
Copy link
Author

byplt commented Oct 16, 2024

Thanks, everything seems to be working fine with the -nodelete switch.

I have a few suggestions:

If you think collision management is important (it’s not for me), you could introduce two different switches (instead of just -nodelete):

-preserve (like -nodelete, but with collision, so it overwrites)
-keep (like -nodelete, but no collisions)

These are just suggestions; I’m happy with -nodelete


By the way, does using the -range 0:9999999 switch achieve the same effect as -nodelete?"

Also, I would like to discuss some details about "-input" switch tomorrow.

@mirogeorg
Copy link

I can make the wiki writeable by anyone

I definitely would like to help with docs

@fcorbelli
Copy link
Owner

Wiki is now (should be) open to everyone

@fcorbelli
Copy link
Owner

fcorbelli commented Oct 18, 2024

Thanks, everything seems to be working fine with the -nodelete switch.

I have a few suggestions:

If you think collision management is important (it’s not for me),

(...) not at all

By the way, does using the -range 0:9999999 switch achieve the same effect as -nodelete?"

???

Also, I would like to discuss some details about "-input" switch tomorrow.
Ask, I'm ready to release the latest build

@byplt
Copy link
Author

byplt commented Oct 18, 2024

"-input" has great potential and is quite practical. However, we need a switch that provides destination paths, similar to the -to switch.

Example scenario;
111

With the new switch;
22

Also;

with "-find" and "-replace" switchs, we can't achieve this scenario in one version, and the growth issue occurs.
"-to" switch doesn’t work with "-input" and can’t set paths individually.

Note;

  • What is "the growth problem"? It refers to the archive's file list growing because the same files are being archived repeatedly. The archive data itself won’t increase due to deduplication if the data is identical.
  • Don't mind my "-range" switch question.

@fcorbelli
Copy link
Owner

Please test the pre-release 60.8j (via zpaqfranz upgrade -force)

The growth problem concerns the ever-increasing possibility of name collisions, having essentially a potentially flat namespace. Incidentally, the list of files is stored, in blocks i, even if they are completely deduplicated (i.e., the data fragments are not written to the data blocks)

@byplt
Copy link
Author

byplt commented Oct 18, 2024

Two issue exists;

  • In the scenario, chatlogs.txt is not processed. I set its destination as a folder (you can see it in the scenario image), so the filenames shouldn’t matter for a folder destination.

Output;
image

  • Secondly, I added a new photo as 4.jpg, and it seems to have created a new folder. Is that expected?

Output;
image

@byplt
Copy link
Author

byplt commented Oct 18, 2024

Actually, these issues aren't very important. I think it would be fine if it worked this way.

@byplt
Copy link
Author

byplt commented Oct 18, 2024

You can restrict, the destination of a file input, must be file path.

@fcorbelli
Copy link
Owner

You can restrict, the destination of a file input, must be file path.

Ahemm... No
It's up to the user, if really want to mix some weird/strange options

@byplt
Copy link
Author

byplt commented Oct 18, 2024

Thanks for your effort. I have a suggestion:

For the "-x" switch, you could implement the reverse of these operations.

For example:

zpaqfranz x archive.zpaq -output "outputfiles.txt" -destination "destinations.txt"

This is just a suggestion; I don’t need it.

@fcorbelli
Copy link
Owner

fcorbelli commented Oct 18, 2024

Thanks for your effort. I have a suggestion:

For the "-x" switch, you could implement the reverse of these operations.

For example:

zpaqfranz x archive.zpaq -output "outputfiles.txt" -destination "destinations.txt"

This is just a suggestion; I don’t need it.

Ahem... no
-destination will continue as multiple to
-input as multiple files

BTW you should to file to file, and folder to folder

filesandfolders.txt

c:\games\x\savegames\save1.dat
e:\apps\whatsapp\chatlogs.txt
c:\myfiles\photos\

destinations.txt
You can't to file to folder, or vice-versa

SaveGames\x_game_lastsavegame.dat
wpLogs\chatlogs.txt
myphotos\

EDIT: of course it's up to you the \ handling
If you use \ (in files) you should use \ (in to)
If you don't use , you should NOT use \

c:\games\x\savegames\save1.dat
e:\apps\whatsapp\chatlogs.txt
c:\myfiles\photos\

This is NOT good

SaveGames\x_game_lastsavegame.dat
wpLogs\
myphotos

wpLogs\ is a folder, but e:\apps\whatsapp\chatlogs.txt is a file
c:\myfiles\photos\ ends with \ but myphotos don't

@byplt
Copy link
Author

byplt commented Oct 18, 2024

Thanks, there don’t seem to be any other problems. I didn’t realize that the "-input" and "-destination" switches also work for "-x".

@fcorbelli
Copy link
Owner

They works everywhere

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants