Can't create symlinks for files on other drives on Windows 10

changed the description

I think this might be similar: jami-client-android#1053 (closed)

However, for me (Windows 10 -> "real" Android smartphone (no emulator) always SAME user account (myusername -> myusername ) it NEVER works

Also similar?: https://forum.jami.net/t/cant-copy-5gb-file-from-jami-fdroid-to-linux-desktop-jami-202112162104/1094

Yes I think our issues might be linked indeed! Thank you for sharing it, I probably should have looked a bit deeper for similar issues before posting mine..

Interesting how you found that sending the file from an android version emulated within windows still has the issue. Could mean that the issues is linked with the OS handling the data transfer and not the actual Jami version used? Don't know if anybody tried it on Linux too

Could mean that the issues is linked with the OS handling the data transfer and not the actual Jami version used?

To be honest I don't know. But I think it's an issue regarding Swarm (old versions Non-Swarm <-> Non-Swarm) never had this issue - at least since I am using Jami.

My issue is probably related to: jami-project#1391 (closed) (However - I could not reproduce if I delete my own account from the contact list using a "real" smartphone)

My issue is probably related to: jami-project#1391 (closed) (However - I could not reproduce if I delete my own account from the contact list using a "real" smartphone)

I don't see how the file transfer issue is linked to this issue? I tried deleting my own account from my contact list and nothing changed. Does having yourself as a contact change anything about your other conversations?

Forgot to mention that the client receiving the transfer does not seem to matter as I'm having the same issue when transferring files between two Windows 10 clients, (Windows 10 Pro 21H2 and Windows 10 Home 20H2).

So it seems that any big file (> ~100Mb) sent from a Windows 10 client on version 202112221635 will show as incoming on the machine hosting the file and the transfer will fail to initialise.

I believe transferring files depends on the daemon and NOT on the client ... but I might be wrong.

So the OS should not (?) matter?

That is going further than my current understanding in OSs. But if the issue is in a specific daemon, for instance the Windows Jami daemon, wouldn't the issue not occur when running Jami in an emulator? As the daemon being used there would be the android one?

I think only developers such as @sblin can answer and fix this issue.

EDIT:

But if the issue is in a specific daemon, for instance the Windows Jami daemon, wouldn't the issue not occur when running Jami in an emulator?

Using Bluestacks 5 (Android Emulator, newest Android-Version of Jami): jami-project#1391 (closed)

It is possible to send a file Android Emulator (Bluestacks 5) -> Android (Mate 20 Pro) if I use the first account. However if I use the other account I just sometimes get an invitation and it fails.

Can you provide the logs while reproducing the issue? https://git.jami.net/savoirfairelinux/jami-project/-/wikis/tutorials/Bug-report-guide#client-qt-windows-gnulinux (Ideally both sides, but at least the sender with the problem)

I don't have any windows 10 nor android emulator today, but on android & GNU/Linux I can transfer files without issues (I have some files of 50Gb).

The logs should show something. And I don't see where the limitation can be

cc @agsantos @atraczyk if you can check for windows 10?

I think I found the resolution (at least regarding Windows 10). You have to launch Jami as an administrator and suddenly (Windows 10 -> Android) works.

However - it should be noticed that you were able to transfer files using Jami before the last (Android?? or Clientqt) updates WITHOUT having to lauch Jami as an administrator. So it's a new issue.

@Detarmony Can you try to transfer files over ~100 Mb if you launch Jami as an administrator?

My issue is related to: jami-client-android#1053 (closed)

I got the resultion because of the logs (Windows 10,Version 202112221635; if I don't launch Jami as an administrator):

[1644912425.306|14800|fileutils.cpp :350 ] Couldn't create hard link: create_hard_link: Das System kann die Datei nicht auf ein anderes Laufwerk verschieben.: "D:/MyUser/Pictures/samplefile.png", "C:\Users\MyUserPath\AppData\Local\jami\934c5d955207febf\conversation_data\c2239139f24aa03d284ebac3a7cfa23f475556eb\c0293fe29505dc577243582a218396258e999cf9_7861972829611311.png" [1644912425.307|14800|fileutils.cpp :333 ] Couldn't create soft link: create_symlink: Dem Client fehlt ein erforderliches Recht.: "D:/MyUserPath/Pictures/samplefile.png", "C:\Users\MyUser\AppData\Local\jami\934c5d955207febf\conversation_data\c2239139f24aa03d284ebac3a7cfa23f475556eb\c0293fe29505dc577243582a218396258e999cf9_7861972829611311.png"

Note: I modified the logs because I replaced my own Windows 10 username and file name. "Myuser", "MyUserPath" and picutures "samplefile".

Hardlink will fail for sure as you use D: and C:

For softlink, seems to be a bug on windows, they don't document this limit and https://docs.microsoft.com/en-us/cpp/standard-library/filesystem-functions?view=msvc-170#create_symlink should take care of administrative flags itself. Not sure to have a solution from our side.

Maybe reuse the old API... https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createsymboliclinka

I'm not sure what hardlink and softlink means. But may I ask: If I launch Jami as an administrator and I follow the exact same steps it seems it does not matter if it is hardlink or softlink ?

So why does it work if I launch Jami as an administrator? I think in the past (Non-Swarm -> Non-Swarm) I never had any issues regarding file transfer (Windows 10 <-> Android) even if i didn't launch Jami as an administrator?

It's a symbolic link (hard links, soft or junctions). Basically to avoid to store a file multiple times. On windows, for a long time, only administrators can create symbolic links. Then they added a flag to create when unprivileged. Here it's seems that >100Mb the administrator level should be there, which is obviously a bug from Windows API.

Comparing non swarm to swarm is a non sense. Non swarm is not synced across devices, without any automatic retry on error. It's not the case since Swarm and symbolic links are used to make syncing available.

Sure! I've got the full logs but I'm trying to remove any personal information before sharing it. However, those two lines from the logs of the device sending the file seem to give evidences to what you were saying @sblin :

[1644937924.123| 2300|fileutils.cpp     :350  ] Couldn't create hard link: create_hard_link: The system cannot move the file to a different disk drive.: "H:/archlinux-2021.12.01-x86_64.iso", "C:\Users\deta\AppData\Local\jami\15e92d4ca018687a\conversation_data\1b212e1032320041326110ffc10e1a65c22808cc\3dc01cdde73d834d61e945f0f22310caffa622f4_1318205149949014.iso"
[1644937924.123| 2300|fileutils.cpp     :333  ] Couldn't create soft link: create_symlink: A required privilege is not held by the client.: "H:/archlinux-2021.12.01-x86_64.iso", "C:\Users\deta\AppData\Local\jami\15e92d4ca018687a\conversation_data\1b212e1032320041326110ffc10e1a65c22808cc\3dc01cdde73d834d61e945f0f22310caffa622f4_1318205149949014.iso"

@Detarmony So it does work to transfer files over 100 Mb if you launch Jami as an administrator?

@sblin So the only workaround is to always launch Jami as an administrator? If we do so there should be not limit? Except you once wrote that the partition system won't support a (theoretical) file transfer of files with a size of 10 TB ?

was about to try, but anything I do right now is a bit cumbersome as I have to access my home machine that has the issue remotely The network and my laptop aren't that good

You could try to transfer files using the same wifi hotspot.

Yes, for now it's a workaround (or they may have fix the bug on Windows 11, I don't have any windows device, so can't check)

A proper fix would be to check if CreateSymbolicLinkA have the same issue.

Or a dirty ugly hack to implement our own system of symbolic links or even duplicate the file. But I don't want to see this in the code.

I can try over my local network when I get back home @El4. However, can confirm running the client sending the file as admin works no problem.

But wait, I thought when transferring an file, Jami would copy it to the conversation's folder instead of storing the path to the original file.

Was I wrong?

it's an hardlink when possible, we don't duplicate files, imagine sending movies, you don't want to duplicate 10Gb

so the file that appears in the conversation's folder isn't actually there even though windows shows it as a proper file with a size and all?

I think you have to select "save file"? Right-click.

Do you still want the full log @sblin then?

it's ok, we have the original problem and enough informations, thx

Alright! Thanks for the explanations! Seems it had nothing to do with filesize in the end.

Should I close the issue or leave it until it's fixed? I'm very new to Git

Just one last question ... but I'm not sure if it is related to this topic anmyore:

Android Emulator (Bluestacks 5) -> "real" Android did not work the first time. However, if use a second message with a second file I suddenly get the attachments.

Although it seems to work (sometimes?) do I also need to launch Bluestacks 5 as an administrator? Or is my issue not related to this topic?

@Detarmony we will let the bug open until fixed (the team is really reduced these days, so clearly it will take time)

Seems unrelated, here it's everytime and reproducible

Ok I leave the other topic open.

"Windows 10 Home - 10.0.19043"

The symlink creation fails if we are trying to reference a file in another driver.

The file size here is not relevant since it fails for any size.

There are two temporary and user dependent work arounds:

run Jami as adm;
activate dev mode in your machine.

Note: neither of the above options is recommended.

@agsantos May I ask the following questions?

run Jami as adm;

Why is this option NOT recommended? And when will we get a new update of Jami (Windows 10, non-beta version)? Currently I am using Version 202112221635.

Regarding jami-client-android#1053 (closed) : Seems (nearly the same issue).

Do you also need to launch Bluestacks 5 as an administrator? Can you try to reproduce?

If you and @atraczyk have Windows 10 machines can you try reproduce and fix the following bug: https://git.jami.net/savoirfairelinux/jami-product-backlog/-/issues/36

I think a better term would be that we do not suggest. Also, jami should run without the adm and/or devmode.
The issue is too polluted, but if bluestacks is running (or simulates) a different driver that the driver file, so they could be the same issue.
I cannot reproduce your bug between a s10e and windows 10. But please avoid mentioning non related issues in another ticket. This makes our tickets easier to understand if the topic doesn't suddenly changes.

Ok ... Thanks for your answer!

@agsantos @sblin I just used Element (Link: https://element.io/ ) Windows 10 -> Android

I don't have any issues even if I don't launch Element as an administrator. So why is it possible that Element has no issues regarding symlinks but Jami can't transfer files without launching Jami as an administrator?

Hi @El4

I cannot answer for their code, but jami relies on creating symlinks to do a file transfer. In windows (bug or not) one can only create symlinks to a file in another driver if the application is run under adm rights or with the system devmode activated (symlink to files in the same driver are not affected). This is a restriction from the operational system. With that in mind I can only suppose that they do not rely on symlinks to perform their transfers.

Furthermore, based on win32 documentation, we also suppose hardlinks or shelllinks may not have that restriction. The problem with the first, as already stated Sebastien, is that it creates a copy of the file, so a file with 10GB would be copied into Jami appdata to be transferred. Once this ticket is put to development, we will checkout the shelllinks for windows.

I saw Element showed "uploading" but I don't know what that means regarding file transfer ... (maybe they copy the file to another directory first or they upload the file to their server?)

But since Element is open-source you should be able to check: https://element.io/open-source

Would there be another way for Jami to do a file transfer (WITHOUT creating symlinks or using a data server[Jami has no data servers])?

And another question: Previous versions of Jami (Non-Swarm -Non-Swarm) had no issues like that.

Is this caused by an update of Windows 10 or did old version of Jami use another technology?

This is just out of subject. It's not because we use symlinks in our code base that all the software are using this. Element uses a server to store this kind of infos anyway

And again Non swarm was working differently

Comparing non swarm to swarm is a non sense. Non swarm is not synced across devices, without any automatic retry on error. It's not the case since Swarm and symbolic links are used to make syncing available.