Large files on non-NTFS partition

[Malgoth]

Got an issue when trying to download large files with Bitcomet 1.17. When opening a torrent file and choosing destination directory is says: "File to be downloaded is larger than 4GB, please modify your destination location to NTFS...". Okey-dokey! I know that FAT32 doesn't support files larger than 4GB, but the thing is I'm using an EXT3 (for network media player) formatted HDD partition which is connected to Windows XP OS via Ext2Fsd utility. Windows recognizes it correctly (like an ordinary FAT32 or NTFS local drive in "My Computer") and says it is EXT3 partition in its properties.

SO, it would be really nice if in such cases (file(s) >4GB in torrent) Bitcomet could check whether the destination file system really is FAT, FAT32 or any other FS that doesn't support large files and only then give an advice to change FS.

[kluelos]

Filesystems do not generally report their own parameters and limitations. That is, Win98 does not say, "I don't support files larger than 4GB.".

Rather, this is extrinsic knowledge -- you (as a programmer) just have to know that. The way Windows applications do this is to ask what the filesystem is mounted as. It can only handle a few answers. If the answer is not NTFS, then it must be one of the other two filesystems extrinsically known about, which are extrinsically known not to support files this large.

Likewise, since filesystems do not report this, applications do not query about it. One cannot ask, through an OS call, what the maximum filesize supported is. There is no common mechanism to ask, and no common mechanism to answer, this query. Not much need, since most OS's can't even communicate with most filesystems to ask any questions in the first place.

It follows that unless BitComet and other applications are charged with supporting every new and fringe OS in existence (they aren't, they won't, kthxbye), then it is up to the operating system in question to supply answers to the questions applications CAN ask (like, what OS are you?), that allow operations to continue.

Very, very few applications are going to be able to handle an answer like, "I'm Ext2Fsd, thanks for asking!" in a usable way. Unless Ext2Fsd becomes so extremely mainstream that it actually gets Windows support (Mac can't, OS/2 can't unix can't, even Netware can't) for some reason, it's unlikely that most applications ever will support it or other fringe OS filesystems. If Ext2Fsd cannot handle this environment, it cannot handle Windows, nice try, next contestant, please.

It's going to be up to Ext2Fsd to answer, "Yes, I am NTFS", to that question in order for the vast majority of applications not to choke on the answer.

To put it another way, Windows doesn't natively support any of these other file systems, all of which are in far wider use than Ext2Fsd. So hold not thy breath.

BitComet does not work on the requisite level to bypass filesystem commands and do raw disk access for other filesystems. It uses Windows to do that, and the developers are unlikely to rewrite enough of the basic OS to intervene at those levels. If they wanted to work on filesystem internals they probably wouldn't be working for BitComet. BC will be able to do it when Windows is able to do it. Windows still hasn't natively gotten around to NWFS yet.

[Malgoth]

If the answer is not NTFS, then it must be one of the other two filesystems extrinsically known about, which are extrinsically known not to support files this large.

That's the way Bitcomet works now and it can go wrong sometimes (well, at least in my case it does :) ). I suggest not to check whether the destination file system for the large file is NTFS and then decide it must be FAT16 or FAT32 only, but instead of this check whether it is FAT16 or FAT32 and if not - allow torrent task to start. In this way nothing is changed for these 3 native Windows file systems but allows in cases like mine to get the things done.

Or at least make some advanced option to ignore this 'Non-NTFS'-warning...

BitComet does not work on the requisite level to bypass filesystem commands and do raw disk access for other filesystems. It uses Windows to do that, and the developers are unlikely to rewrite enough of the basic OS to intervene at those levels. If they wanted to work on filesystem internals they probably wouldn't be working for BitComet. BC will be able to do it when Windows is able to do it. Windows still hasn't natively gotten around to NWFS yet.

Thats what I'm trying to say! No need for Bitcomet to get raw disk access or something - with Ext2Fsd Windows handles EXT2/EXT3 partitions as easy as its native FAT/NTFS.

No need for Bitcomet event to know about the Ext2Fsd running - just start the torrent task and Windows will store the files on this partition just perfectly!

[kluelos]

That's the way Bitcomet works now and

No, that's the way Windows works now. BC just makes calls into the windows file system at a much higher level. It does NOT make low-level OS calls or attempt to bypass same.

No need for Bitcomet event to know about the Ext2Fsd running - just start the torrent task and Windows will store the files on this partition just perfectly!

BitComet does not know.

BitComet only knows what it's told by Windows, namely that this is not an OS which supports large files.

BitComet doesn't make that judgment call. Windows does. BC proposes to write a file larger than 4GB to the disk, and the operating system returns an error, which BC interprets for you as the message you saw.

BitComet does not get involved in filesystem operations on that level.

It is going to be up to Ext2Fed to work out an accomodation with windows, to present itself as NTFS in order to work around this. BitComet cannot adapt to it short of bypassing the Windows disk handling and taking care of that itself. This isn't going to happen.

Ext2Fed must change, must adapt to Windows. Windows isn't going to adapt to it. You are wasting your time trying to pursue this as a change to BitComet.

[Malgoth]

No, that's the way Windows works now. BC just makes calls into the windows file system at a much higher level. It does NOT make low-level OS calls or attempt to bypass same.

To me the conversation between BC & Windows is like this:

BC: Large file in the torrent. Hey, Win, what's the file system of the destination partition and is there enough free space?

Win: FS is EXT3, enough free space.

BC: <_< uh... what's that? Oh, never mind, you Win can only handle large files with NTFS and this must be some kind of a FAT16 or FAT32 :D

Win: Well, actually it's not, and I can...

BC: No, you can not, Win B) Hey, user, give me NTFS for this torrent!

And some prove is on the way...

For example, Total Commander (and any other Windows file manager) works with Ext2Fsd-mounted EXT3 partition and large files on it just perfectly (I:\ is an EXT3 partition):

But let's try something more relevant to BitComet like µTorrent :rolleyes:

Adding a torrent with a large file to a EXT3 as a destination:

Starting the task:

Works OK... Strange, isn't it? :rolleyes:

Checking the file in the destination dir:

No errors, no "give-me-NTFS" demands, works just as it should.

The very same torrent in BC - and you already know what the result will be!

BitComet doesn't make that judgment call. Windows does. BC proposes to write a file larger than 4GB to the disk, and the operating system returns an error, which BC interprets for you as the message you saw.

But why doesn't OS return any errors to Total Commander, µTorrent or any other application? Or would you say they all perform that low-level disk access bypassing Windows HDD managment? :lol: There would be no need for Ext2Fsd then.

BitComet cannot adapt to it short of bypassing the Windows disk handling and taking care of that itself.

There's no need for this! All the troubles come just of BC's logic: "Large file on non-NTFS? Won't even try and ask Windows if it can handle this!"

[Malgoth]

Maybe I'm wrong with my inferences ;)

So, any suggestions why µTorrent works OK in this situation and BitComet doesn't? :wacko:

[kluelos]

µtorrent simply doesn't ask. (You don't have to.) And when it doesn't work, µtorrent cheerfully runs off the rails.

I hung on to Win98 for a long time past its sell-by date, on a dual-boot system. (I still dual-boot Win2k).My largest partition then was FAT-32, and I started µtorrent to downloading the several DVD iso's of Myst: Uru. The disk images were > 4GB, but I'd forgotten about that limit. µtorrent cheerfully started the download, then just abruptly choked with a disk error when the file tried to grow larger than 4GB.

µtorrent left it to me to figure out why it had just stopped and couldn't write to the disk. (It was this incident that finally got me to embrace XP, though 2K was still my primary OS. Funny, but I didn't even like Uru as it turned out.)

They're both valid (for some value of valid) ways of handling the issue. It's your choice, but I druther have BitComet's way, "doesn't look like this will work", than µtorrent's stoner-like, "it broke, d00d, dunno what's wrong...".

[Malgoth]

µtorrent simply doesn't ask. (You don't have to.) And when it doesn't work, µtorrent cheerfully runs off the rails.

That's not true. For the latest stable release version 1.8.5 if you try to start a >4Gb-file task on a FAT32 partition, you'll get this:

I hung on to Win98 for a long time past its sell-by date, on a dual-boot system. (I still dual-boot Win2k).My largest partition then was FAT-32, and I started µtorrent to downloading the several DVD iso's of Myst: Uru. The disk images were > 4GB, but I'd forgotten about that limit. µtorrent cheerfully started the download, then just abruptly choked with a disk error when the file tried to grow larger than 4GB.

µtorrent left it to me to figure out why it had just stopped and couldn't write to the disk. (It was this incident that finally got me to embrace XP, though 2K was still my primary OS. Funny, but I didn't even like Uru as it turned out.)

That could happen with the early versions of µt, but from that days they have worked on this issue for sure :)

They're both valid (for some value of valid) ways of handling the issue. It's your choice, but I druther have BitComet's way, "doesn't look like this will work", than µtorrent's stoner-like, "it broke, d00d, dunno what's wrong...".

This is where we come again to your previous statement:

BitComet doesn't make that judgment call. Windows does. BC proposes to write a file larger than 4GB to the disk, and the operating system returns an error, which BC interprets for you as the message you saw.

I just tried to copy a 5Gb image file in Windows Explorer (I mean this native file manager) to a FAT32 partition with enough free space and, as expected, got this error

That's the error you are talking about. But there aren't any errors if you try to copy it to NTFS or EXT3 partition in the same Windows Explorer!

And all I'm asking is to really make BitComet work the way you described above: "BC proposes to write a file larger than 4GB to the disk, and the operating system...". Let BC offer the OS to write a large file! If the OS returns an error like it does in case of FAT32, then stop, but if it doesn't - let the OS do the job!

[gavin.wang]

Thank you. It will be fixed in v 1.18 or v1.19 .

[Malgoth]

Thank you. It will be fixed in v 1.18 or v1.19 .

Oh, thanks! That would be great.

kluelos :P

[kluelos]

You're welcome, Malgoth.

[meanie]

Is this fix?  I still have the issue with BitComet not able to save to a network drive with the latest version.  It's requiring NTFS even though the network drive is NTFS formatted and other torrent programs can write large files to the drive.

[Rhubarb]

You do realise you are asking a question to which the last posting was 9 YEARS ago?

[eefaan]

On 12/30/2019 at 8:18 AM, Rhubarb said:

You do realise you are asking a question to which the last posting was 9 YEARS ago?

However I still meet the error on version 1.63.0 (macOS). When I modify the properties and click the 'OK' button, BitComet give me a 'tip' says 'The file to be downloaded is larger than 4GB, please modify your destination location to NTFS disk drive!'. My disk is exFAT format, btw.

Otherwise, sometimes I didn't meet the tip even if the file is almost 6GB, and the file can be downloaded as expect, which is strange.

 

[Rhubarb]

While theoretically Ex(tended)FAT canhandle files up to 128 Petabytes, it's possible that the Mac version of FreeBSD is limiting the amount that can be transferred. Noit being a Mac person (well not since they switched to Intel CPUs and the Marklaar engine under the desktop) I can't give a specific answer

By the way, necro-posting is a very bad idea -  start a new thread of your own if you have issues rather than a 'me too' on a thread that has been dead for years

[wxhere]

On 2/17/2020 at 10:59 AM, eefaan said:

However I still meet the error on version 1.63.0 (macOS). When I modify the properties and click the 'OK' button, BitComet give me a 'tip' says 'The file to be downloaded is larger than 4GB, please modify your destination location to NTFS disk drive!'. My disk is exFAT format, btw.

Thanks for feedback. This bug will be fixed in next release of BitComet macOS edition.

[meanie]

On 4/9/2020 at 12:47 AM, wxhere said:

Thanks for feedback. This bug will be fixed in next release of BitComet macOS edition.

Can you make a general comment regarding saving large files to any non-NTFS locations (such as network drives/shares or other FS)?

I have used BitComet for a long time and looks like I need to switch to another torrent program as more and more files are larger than 4GB, and I'm saving to non-NTFS locations that supports large files, but BitComet refuse to use those locations; while other programs work fine.

[Rhubarb]

As stated, the devs are working on that

[meanie]

On 4/9/2020 at 12:47 AM, wxhere said:

Thanks for feedback. This bug will be fixed in next release of BitComet macOS edition.

Will you fix the issue of saving 4TB files to any non-NTFS locations (such as network drives/shares or other FS)?  Not specifically related to issues with macOS or exFAT?  This issue has existed for 10 years, though it says it will be fixed in 1.18 or 1.19, it remains an issue.

[Rhubarb]

The question has been answered by one of the devs.

THEY ARE WORKING ON IT

[meanie]

What's the answer?  Specifically addressing exFAT on MacOS is not going to fix this.  This bug existed for 10 years?

[Rhubarb]

As I said - there will be an answer when they have one - as of now, they are working on it.

Repeatedly asking for an 'answer' to something that is still being resolved is simply wasting your and everyone else's time. When it's ready, it will be released.

[pgd5776]

I get this problem with smb folder in windows system. I have a solution. I use the "mklink /D" command to make a softlink dir in a local ntfs disk. BItcomet say nothing.😉

Edited October 2, 2022 by pgd5776 (see edit history)

[SlvrSrfr]

yes fairly certain it has happened to me as well. Along with two external(s) failures both of which were brand new.

[IceShroom]

Same issue here.

I'm using a win10 virtual machine and host is a linux system. I'm try to download file bigger than 4g to a shared folder (linux host share to  win10 virtual machine).

bitcomet said :"File to be downloaded is larger than 4GB, please modify your destination location to NTFS...".

okay,

I create a symbolic link to my Shared folder, and Manually typing the address into the address box, because if you just select your symbolic link address, bitcomet will known it was a soft link and parse the real address to address box. To bypass it I have to manual typein address.

In my opinion, The software should use the return value of the CreatFile function to determine whether the current file system supports 4g files，the software should try to create file first, If every thing is okay, just keep working.