How to convert DOS/Windows newline (CRLF) to Unix newline (LF) in a Bash script?

When transferring files between Windows and Unix systems, you'll often encounter issues with end-of-line characters. Windows uses CRLF (Carriage Return + Line Feed) while Unix uses LF (Line Feed) only. This guide shows three methods to convert Windows line endings to Unix format using Bash.

Understanding the Problem

Windows files use \r
(CRLF) for line endings, while Unix systems use
(LF). When viewing Windows files on Unix, you may see ^M characters at the end of each line.

Method 1: Using dos2unix

The dos2unix command is specifically designed for this conversion and comes pre-installed on most Unix systems ?

# Convert file in-place
dos2unix filename.txt

# Create a converted copy
dos2unix -n filename.txt converted_file.txt

# Convert multiple files
dos2unix *.txt

Example

# Check line endings before conversion
file windows_file.txt

# Convert the file
dos2unix windows_file.txt

# Verify conversion
file windows_file.txt

Method 2: Using sed

The sed command can remove the carriage return characters (^M) that appear at line endings ?

# Remove carriage returns and save to new file
sed 's/\r$//' windows_file.txt > unix_file.txt

# Convert in-place using -i option
sed -i 's/\r$//' filename.txt

Example

# Create a file with Windows line endings for testing
echo -e "Line 1\r\nLine 2\r\nLine 3\r" > test_windows.txt

# Convert using sed
sed 's/\r$//' test_windows.txt > test_unix.txt

# Check the difference
wc -c test_windows.txt test_unix.txt

Method 3: Using tr

The tr command can delete carriage return characters from the file ?

# Remove carriage returns
tr -d '\r' < windows_file.txt > unix_file.txt

# Using with cat
cat windows_file.txt | tr -d '\r' > unix_file.txt

Example

# Convert and display character count
echo "Original size:"
wc -c windows_file.txt

echo "After conversion:"
tr -d '\r' < windows_file.txt | wc -c

Comparison of Methods

Verifying the Conversion

You can verify successful conversion using these commands ?

# Check file type
file filename.txt

# Display line endings visually
cat -A filename.txt

# Count characters (CRLF files will have more)
wc -c filename.txt

Conclusion

Use dos2unix for dedicated line ending conversion, sed for flexible text processing with in-place editing, or tr for simple character removal. All methods effectively convert Windows CRLF line endings to Unix LF format.