The best way to learn linux command line is to use it!
One of my favorite usage is
ls -l -t
It lists files ordered by modified time, with detailed file information.
To cut some filds (columns) from a table-like file (.csv file for example).
For example, you have a csv file (dump.csv) with 15 columns, the first few lines
look like this:
to cut column 3 to 15 out of the file, simply type:
cat dump.csv | cut -d ',' -f3-15
One of the most useful command for file processing.
In my experience (data mining experiment), one frequent situation is
to paste the true labels to the estimated posterior, then use the new
file to calculate AUC.
paste -d '\t' labels posteriors > combination
An program for calculating AUC will be given later.
A common linux command, I like to append test data to training data using this command.
Shows the full path of a command. For example, when writing python scripts to be
executed, you need to put
at the beginning of the script file. If you don't know where is python, you can
the above path will show up.
Sort entries of a file in some order according to some key columns. Some of
the most useful options:
-k Specifying which column you like to treated as key for sorting
-n Treat the key as number rather than string
-g When the keys are in scientific notation, you need this option to
turn those strings into correct numbers
-r Sort in reverse order, the greater ones will come first
For huge file, even for VIM, it takes a while to open. Sometimes you don't want to see
that whole file, you just need to check the first and last few lines. That's when
these commands come into play. Try these:
head -n 10 huge_file
tail -n 10 huge_file
To replace some characters in a string with another character.
- Traverse all files under one folder
When you want to run your code on different files which reside in different folders, then
you need to visit these files, but you don't want to type in a lot of similar commands
to give one shot for each file.
for f in *
- Testing file properties
Some useful ones, for a complete list, refer to test
Testing file existence
Testing if file is a directory