I have observed one recent issue about ftp from windows to Linux.
Initially we had Solaris unix and any file sent from windows to solaris via ftp (binary or ascii) worked smoothly.
Say for e.g if I sent a shell script to soalris from windows, that script used to run corectly. So did an encrypted file which got successfully decrypted on Solaris.
Things changed when we got Redhat linux which replaced solaris and now all the above file transfers have caused issues. In that, once the file was successfully transferred to Linux, and if you open the file in Vi editor, you will see [dos] at the bottom. This is not about ^M characters. I assure that files are sent in correct mode. So we dont get ^M chars. Only the [dos] at the bottom when opened in Vim. so the shell scripts having [dos] at the bottom dont run and error out.
The binary files sent via ftp also dont work when ran on Linux.
Thus, all these files run only when dos2unix is ran on them. On text files the [dos] symbol goes away and bainary files get run successfully then.
My question to experts here is, why in solaris, a file which is sent in correct mode from win to solaris does not need a dos2unix processing and why those files need dos2unix processing when sent from win to Redhad or Oracle Linux.
Most probably because Solaris ftp had a default mode of "binary" whereas Linux ftp has "ASCII" or thereabouts.
ftp (the command, not the protocol) has two modes: "binary" sends files as they are. "ASCII" becomes aware on which system type it runs and translates line-ending sequences to the respectively correct sequence on the target system. I have written a lengthy explanation about what is going on and why which you can read if interested.
Perhaps the Linux ftp can be configured to do the same as the Solaris client did.
But, the mode is not the issue. I have sent the same file from win to Linux in both, binary and ascii modes. In both cases [dos] sign att he bottom of the file when received at linux end says that it has some dos chars.
Sorry if I have missed your actual point which you wanted to convey. But, modes are not the issue as in both the modes we have to run dos2unix on the said file when received on linux.
What do you mean by "[dos] sign" this is definitely not a diagnostic message of the ftp command, yes?
Of course they are, but you have probably not yet identified the culprit. The only thing "dos2unix" and similar tools do is to replace <CR><LF>-styly line ends (DOS/Win) to <NL>-style line ends (UNIX). Use a hex editor on what you call a DOS-file and on what you call a UNIX-file and you will see this exact difference described in the post i linked to above.
Create a test file (in UNIX) of two lines, like this:
AB
CD
and use this command in UNIX to display it:
# od -ax /path/to/testfile
then transfer it, so that it is converted to "DOS" mode, then do the same again. You will see the difference easily.
To my knowledge the [dos] tag in vim says that there are CRLF line endings and the ^M sequences are not displayed.
I remember some time ago the vsftp FTP-server on RedHat ignored ascii mode and always used binary mode, no matter what the client set.
I agree with cero. Redhat says it's doing a ASCII mode transfer but it lies and does a binary mode transfer anyway. I think that you have to force this. Please post the content of /etc/vsftpd/vsftpd.conf
A recent vim, when the majority of lines have <CR><LF>, does not display the <CR>s as ^M, but goes into [dos] mode. New lines will get the extra <CR>. So you can directly edit DOS files.
--
DOS (United States) Department of State
DOS Denial Of Service [attack]
DOS (Microsoft) Disk Operating System
One quick question, you have mentioned here that Redhat is doing ASCII transfer, but the issue we face is when I transfer the file from windows to Linux and not Vice Versa.
Also,
as mentioned previously, in the command window of dos, during ftp tp linux, I had mentioned ascii. So the ftp client on dos must have sent the file in ascii mode OR here the target(Linux) FTP server dictated terms and forced in binary?
Regarding contents of /etc/vsftpd/vsftpd.conf, I cannot put it here due to restrictions but, can you please tell me what exact thing to look into it?
---------- Post updated at 10:52 AM ---------- Previous update was at 10:42 AM ----------
One more thing, even if Redhat Linux forces the use of Binary mode, I also faced the issue when sending a proper binary file to Redhat. It also had the dos chars. Hows that possible then?
Not quite, I'm afraid. Back in those days when you had DOS PCs and code pages like 437 or 850, I had a long sed script to convert extended DOS char sets to *nix local char sets, which later was done by dos2unix on the fly (if I remember correctly, that is). It also converts between UTF-8 and UTF-16, I hear say.
But nowadays, when windows uses UTF-8 as well, it may reduce to just removing the <CR> characters.
Yes, found these line, but they are commented. So I guess its forcing binary as you said. But will these settings, which hare on the target Linux box, have any effect on the ftp which we do from a windows machine to it?
Yes, sure they can. The config determines how the whole service behaves, in both directions. I am guessing but I reckon the Redhat default setup is different to the Solaris setup. You haven't changed anything on the Windows end have you? You've just changed the Unix/Linux end which infers the Linux end has caused the problem.
Remember though if you change any of the config that you need to restart the service for it to take effect. Be professional and take a copy of the whole damned thing before you mess with it.
No changes on the windows side. But on Linux file I see that it forces binary as it fears denial of service attack. Not sure why Solaris didnt fear it, so may be it had some other protection.
So now what solution can we have for it?
Also the fact that Redhat is forcing binary is still not answering the other question as to why even files such as encrypted files which are sent from win to redhat in binary mode (explicitly chosen) still has dos chars. When we run dos2unix on them the size decreased and decryption happened successfully. Initially I had thought that those files were sent on Redhad in ascii mode by mistake. But now it is clear that they indeed went in binary (due to forcing). Then how they had dos chars issue?
As I already said, the vsftpd.conf file determines how ftp behaves (and it can be quite a large file).
If you can't post the contents here then I think you are into researching the configuration options yourself. At least you know where to look.
Do appreciate that it's difficult for us to chip in with the answers when we aren't given all the information.
I'm not a RedHat expert. If you don't get any fairly immediate answers then perhaps ask moderators to move this thread to the RedHat specialist forum where it will more likely be noticed by RedHat specialists. There are already members subscribed to this thread who are also moderators with the clout to make that happen and who will read this anyway.