The logs I am processing is Hadoop log (log4j). It is in format like: 2014-09-20 21:55:11,855 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated user map size: 36 2014-09-20 21:55:11,863 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated group map size: 55 2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Update cache now 2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Not doing static UID/GID mapping because ‘/etc/nfs.map’ does not exist. Now, I
Read more
Tag: Command
How to read email in Maildir on Linux?
Posted onHow to read email in Maildir on Linux? You can use mutt by: mutt -f /path/to/mail/dir/
How to find and change a user’s UID on Linux?
Posted onHow to find and change a user’s UID on Linux? Find the UID of a user: id -u $USER_NAME Change a user’s UID: # usermod -u $NEW_UID $USER_NAME
How to monitor temperatures of laptop on Linux
Posted onHow to monitor temperatures of laptop on Linux? This works on Linux Mint: sudo aptitude install lm-sensors hddtemp For lm-sensors, first detect the sensors by: sudo sensors-detect To detect the temperature in the system: sudo sensors To detect the HDD (e.g. sda) temperature: sudo hddtemp /dev/sda An example of the output: [zma@mini:~]$ sudo sensors acpitz-virtual-0
Read more
Manage Linux console screen by commands?
Posted onHow to manage Linux console screen by commands? When the screen will be blanked? Put the screen into powerdown mode or power off the screen? 2 tools are useful for managing the console screen on Linux: setterm – set terminal attributes.vbetool – run real-mode video BIOS code to alter hardware state. When the screen will
Read more
How to boot Linux Mint to the console by default?
Posted onHow to boot Linux Mint to the console by default? (that is, the run level 3). 2 methods introduced here to boot Linux Mint to console/command line. Method 1 (may only work for earlier releases; check method 2 below): set kernel option. Edit /etc/default/grub and add text to GRUB_CMDLINE_LINUX_DEFAULT: GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash text” Then, re-generate grub
Read more
How to rotate a .mov video from iPhone by 90 degree on Linux?
Posted onThe video files copied from iPhone is rotated by 90 degree when they are played on Linux. How to rotate a .mov video from iPhone by 90 degree on Linux? Specifically: 1) Play the video by rotating 90 degree. 2) Edit the video by rotating 90 degree and save it. You may check Rotating Video
Read more
How to turn a BIOS-based Linux Mint 17 installation to UEFI booting?
Posted onI installed Linux Mint 17 in the traditional BIOS mode. However, Windows 8.1 on my computer can only boot in the UEFI mode. I do not want to reinstall the Windows or erase it. How to turn a BIOS-based Linux Mint 17 installation to UEFI booting? You may refer https://help.ubuntu.com/community/UEFIBooting for some information.
Finding Which Package Provides a File in Ubuntu Linux and Linux Mint
Posted onHow to find which package provides a file in Debian based releases, such as Linux Mint, Ubuntu? You can use `dpkg`: $ dpkg -S /path/to/the/file -S or –search is the option to make dpkg do a “search”: -S, –search filename-search-pattern… Search for a filename from installed packages. To search which package provides a command file
Read more
How to view a file at a specific commit in git?
Posted onHow to view a file at a specific commit/revision in git? You can use git show to view a file’s content at a specific commit in git: https://www.systutorials.com/docs/linux/man/1-git-show/ $ git show REVISION:/path/to/file You can also save a copy by $ git show REVISION:/path/to/file >file.copy
How to set the number of mappers and reducers of Hadoop in command line?
Posted onHow to set the number of mappers and reducers of Hadoop in command line? Number of mappers and reducers can be set like (5 mappers, 2 reducers): -D mapred.map.tasks=5 -D mapred.reduce.tasks=2 in the command line. In the code, one can configure JobConf variables. job.setNumMapTasks(5); // 5 mappers job.setNumReduceTasks(2); // 2 reducers Note that on Hadoop
Read more
How to find which hard drives a LVM volume uses?
Posted onHow to find which hard drives a LVM volume uses to decide which volume will be affected if a disk failes. You can use lvdisplay with the –maps option to display the the mapping of logical extents to physical volumes and physical extents: # lvdisplay –maps To map physical extents to logical extents, use #
Read more
Filter away non-printable ASCII characters on Linux?
Posted onSometimes the commands’ output have some non-printable ASCII characters. How to filter away these non-printable ASCII characters on Linux? You can consider the problem as “how to leave only printable ascii characters?”. The printable characters in ASCII table are: Octal 011: Tab Octal 012: Line Feed Octal 015: Carriage Return Octal 040 to 176: printable
Read more
How to calculate the average of value in lines of a text file on Linux by a script?
Posted onHow to calculate the average of value in lines of a text file on Linux by a script? The file.txt is like: Run 0: Time used: 22.235331711 seconds. Run 1: Time used: 20.784491219 seconds. Run 2: Time used: 21.851638876 seconds. What I want it to calculate the average time. awk is handy for this purpose:
Read more
Setting sbt log level at the command line
Posted onsbt‘s log level can be set by execute set logLevel := Level.Error inside sbt. How to set the sbt log level at the command line? Add –error to sbt for specify the logLevel to error: sbt –error “your command”
How to install multiple versions of sbt on my Linux host?
Posted onHow to install multiple versions of sbt on my Linux host For example, some projects use 0.12.x while some use 0.13.x. Installing neither in the system only is not sufficient enough. You may use the excellent sbt-extras: https://github.com/paulp/sbt-extras Most of the time, it detects the version of sbt needed in the project direoctory automatically: [zma@office
Read more
How to package a Scala project to a .jar with sbt?
Posted onHow to package a Scala project to a .jar file with sbt? Manually packaging the .class files are fine but too tedious. A jar of just the project classes The command: sbt package will produces the main artifact as a jar into target/scala-2.x.y/project_name_2.x.y-zz.jar. Continuously building the package: sbt ~package Standalone jar with all dependencies If
Read more
How to get the time at millisecond level on Linux with command line?
Posted onI know that gettimeofday() is a nice API. But how to get the number of seconds with milliseconds since the epoch time? You can get the time at nano-seconds level (although it is not guaranteed that the last digits are accurate) by: date +%s.%N
How to catch the kill signals sent by kill in C on Linux?
Posted onHow to catch the kill signals sent by the kill command in C programs on Linux? For example, I have a daemon progd running, and will kill it by: pkill progd The question is how to catch the signal sent by the kill command and response to it in the progd program implemented in C
Read more
Running Chrome over SSH tunnel
Posted onHow to run Chrome on remote host over a SSH tunnel? This way, I can access resource that can only be accessed inside the remote host’s network. Running Chrome over a SSH tunnel is much easier than running Firefox over SSH from a Linux host: First, ssh to the remote host with -X option: ssh
Read more