-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Preserve items' timestamp upon upload and download. #13
Comments
Yep, this would be a good improvement to add, although I think it is better to specify it when uploading rather then upload and then patch. Looking at the Graph docs, changing the body sent within the body = {
"item": {
"@odata.type": "microsoft.graph.driveItemUploadableProperties",
"@microsoft.graph.conflictBehavior": conflict_behavior,
"name": "file name.txt",
"description": "String",
"fileSize": 1024,
"fileSystemInfo": {
"@odata.type": "microsoft.graph.fileSystemInfo",
"createdDateTime" : "datetime",
"lastModifiedDateTime" : "datetime"
}
} I haven't tried this and obviously there are some issues to consider such as the "datetime" having to be UTC. Modification time is fairly easy using last_modified = os.path.getmtime(path_to_file)
if platform.system() == 'Windows':
creation = os.path.getctime(path_to_file)
else:
stat = os.stat(path_to_file)
try:
creation = stat.st_birthtime
except AttributeError:
# Likely using Linux, fall back to last modified.
creation = stat.st_mtime If this idea is proceeded then we should probably get rid of the standard |
I have created a new branch to implement this. So far I can get the creation and modification timestamps of the file into the correct format but I can't get Graph to accept the data. Oddly when item is in the json Graph flat out rejects it and if you don't include the item and move everything up a level then Graph appears to just ignore the fileSystemInfo. Maybe you have any ideas on how to solve this? |
Well I tested your code and works well. The format you chose seems to work: # Create request body for the upload session
body = {
"item": {
"@odata.type": "microsoft.graph.driveItemUploadableProperties",
"@microsoft.graph.conflictBehavior": conflict_behavior,
"name": destination_file_name,
"fileSize": file_size,
"fileSystemInfo": {
"@odata.type": "microsoft.graph.fileSystemInfo",
"createdDateTime": file_created_str,
"lastModifiedDateTime": file_modified_str,
},
}
} The only problem I had was related to the fact that the files I uploaded had the creation date set later than the last modified date - this on my local machine, they were files I had manipulated via A part from that (and a problem with httpx not being the last version on this Laptop, which prevented download from working), your code works like a charm, as far as I can tell. |
I added this code after line no. 369 to get more infos from
|
I fixed my issue by removing some of the unnecessary content in the body. Since the Graph API integrates with OneDrive Personal, OneDrive Business, and SharePoint I have seen some issue where something works for one service but not for others, and I think that was the case here. I have also added the suggested item details printed using verbose, although none of my items seem to have hashes. Can you please test the latest push of the branch works for your uploads? Still need to look into for downloads. |
Sorry for not answering you before, I had a hardware crash and I'm having some problems with backups. I gave a look at the new code and as far as I could understand this issue is resolved.
This point is really interesting. I already made a lot of testing when I was developing the async download code. The biggest issue I met was that it is not possibile to parallelize the upload of the "chunks", since each chunk upload must be completed before the next may start. This broke the legs once and for all to a real asyncronous approach. I.e if an upload from bytes 0-999 is ongoing, I got an API an error when trying to start a second concurrent upload over another connection from bytes 1000-1999. It seems to be mandatory to wait until the first operation completes to start the second upload request. So, while keeping a single connection approach, I tried to switch from syncronous I/O (both disk and network-wise) to async I/O (using Then I tried to mix syncronous and asyncronous operations: It seems that miming a syncronous behaviour using Maybe switching to "real" asyncronous code (like threads) may do the trick, but I did not tried. Looking forward to hear what you think! Regards |
Yes, chunks must be uploaded sequentially in order per the Graph docs:
I came to a similar conclusion, little to no gains from disk i/o. I think there only gains that could be made are if uploading multiple items then you could have a connection for each item. In regards to updating the file metadata of a downloaded file - I have made no progress on this. |
I have a (somewhat) working piece of code that can read and write timestamps under Windows using Anyway I believe that updating a file's creation time under Linux is not possible. Here the docs say:
If The cleanest approach could be to write a tiny library in C or C++ which reads/writes files' timestamps and to use it via In my opinion, all of this work should be accomplished just if a sync process between local and remote depends on it to become possible. I don't know if this is the case. Bye! |
Not currently implementable cross platform. Issue stale. |
Good morning,
I believe it would be nice to keep items' datestamp upon upload (as described here, where it seems to be the right behaviour) and downloads.
The docs here describe the fields
createdDateTime
andlastModifiedDateTime
as read-only properties, so theFileSystemInfo
facet, should be used to set those values. This means every item has twocreatedDateTime
and twolastModifiedDateTime
: the ones intoFileSystemInfo
are set by the client, the other ones are set by OneDrive.Updating an uploaded file's
FileSystemInfo
is quite straightforward, using thePATCH
method (docs)The nasty part is to read/write the file's
createdDateTime
on the local machine. Under Windows, it is possible to read the creation time of a file with Python's internalstat
call (although some filesystems may not respond properly). Under Linux, as far as I know, it is not possible to read the creation time via Python'sstat
call. I managed to read the creation time on Linux in pure Python viactypes
, kernel version >= 4.11 and glibc version >= 2.28 are needed.Writing the creation time, under Windows, is possible via
ctypes
and maybe via Python'sutime
. Under Linux, I believe it is not possible to alter the creation time of a file by any mean.Let me know your opinion!
Regards
The text was updated successfully, but these errors were encountered: