The host os is connected into network by wireless one so how to let its guest os connect network. Take [1] as a reference. I run it successfully with following steps. 1, create /etc/qemu-ifup script and chmod it. 2, start a qemu guest os with command sudo ./qemu/qemu-system-x86_64 -enable-kvm -m 1024 -drive file=marss_dram.qcow2 -vnc 127.0.0.1:0
Read more
Category: Tutorial
How to find the DataNodes that actually store a file in HDFS?
Posted onA file may be splitted to many chunks and replications stored on many datanodes in HDFS. Now, the question is how to find the DataNodes that actually store a file in HDFS? You may use the dfsadmin -fsck tool from the Hadoop hdfs util. Here is an example: $ hadoop fsck /user/aaa/file.name -files -locations -blocks
Read more
How to increase the number of files allowed to be opened on Linux?
Posted onOn my system: $ ulimit -n 1024 Some tools like GATK are aggressive in creating temporary files by creating more than 1000 files under /tmp/. This will cause the program to fail. How to increase the number of files allowed to be opened on Linux? To increase the max number of open files to 10240,
Read more
How to write /etc/fstab entry for –bind mounting?
Posted onHow to write /etc/fstab entry for –bind mounting like mount –bind /home/hadoop/hdfs/store-tmp /home/store/tmp From man 8 mount: Since Linux 2.4.0 it is possible to remount part of the file hierarchy somewhere else. The call is mount –bind olddir newdir or shortoption mount -B olddir newdir or fstab entry is: /olddir /newdir none bind
kernel netfront: Too many frags in Xen VM
Posted onWe set up Xen F19 VMs on a Xen 3.4.3 / 2.6.32.13 xenified kernel. Check here.). However, we find the Xen VM keeps reporting: kernel netfront: Too many frags and skb rides the rocket in the dmsg. This solves the problem (assume in physical server vif1.0 is for the eth0 on the VM): On the
Read more
How to change number of replications of certain files in HDFS?
Posted onThe HDFS has a configuration in hdfs-site.xml to set the global replication number of blocks with the “dfs.replication” property. However, there are some “hot” files that are access by many nodes. How to increase the number of blocks for these certain files in HDFS? You can the replication number of certain file to 10: hdfs
Read more
How to get logs of a specific time range on Linux?
Posted onThe logs I am processing is Hadoop log (log4j). It is in format like: 2014-09-20 21:55:11,855 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated user map size: 36 2014-09-20 21:55:11,863 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated group map size: 55 2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Update cache now 2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Not doing static UID/GID mapping because ‘/etc/nfs.map’ does not exist. Now, I
Read more
What’s the difference between Reliability, Durability, and Availability for data storage system?
Posted onSome important concepts in distributed system like Hadoop distributed file system, Google file system and so on. Answer from http://www.quora.com/Whats-the-difference-between-Reliability-Durability-and-Availability-for-data-storage-system The difference between durability and availability is fairly simple. Durability is about what happens when all power goes out everywhere. Has all data been written to stable storage that doesn’t require power (e.g. disk/flash), in
Read more
How to read email in Maildir on Linux?
Posted onHow to read email in Maildir on Linux? You can use mutt by: mutt -f /path/to/mail/dir/
if (p:string list) = c (is the only element)
Posted onif (p:string list) = [c] then (divide p1 c) showing unbound value c. i want to equalise (if one element in p which is (c:anything)) and use that variable let p = [“ocaml”] let f s = match s with | [c] -> print_endline c | _ -> print_endline “ops” f p It will print:
Read more
Making Hadoop Java process heap larger?
Posted onIn Hadoop 2.5.0, I use ‘ps -aux’ and find the Java process has options: -Xmx1000m However, my nodes have 32GB memory. How to make Hadoop Java process heap larger? In yarn-env.sh, you can find: # For setting YARN specific HEAP sizes please use this # Parameter and set appropriately # YARN_HEAPSIZE=1000 In hadoop-env.sh, you can
Read more
How to capture camera photo/video using MPlayer on Linux?
Posted onHow to capture camera photo/video using MPlayer on Linux? You can watch the video from the camera by MPlayer: # mplayer tv:// Note that you need the root privilege to play the tv://. You can use mplayer to capture photos by: # mplayer -vo png -frames 3 tv:// If will store files 0000000{1..3}.png in the
Read more
How to view .mbox files on Linux?
Posted onHow to view .mbox files on Linux? Thunderbird can read mbox files well. You can drop your mbox files to the Local Folders directory of Thunderbird and restart Thunderbird. It will index the emails and show them. To find the Local Folders directory of Thunderbird: Open “Account Settings” and click on “Local Folders” to see
Read more
How to make Thunderbird check for new messages in all IMAP folders?
Posted onHow to make Thunderbird check for new messages in all IMAP folders? Under “Edit/Tools > Options > Advanced”, click the “Config Editor” button. In the config editor, set the mail.check_all_imap_folders_for_new configuration to “true“.
Finding All Available Versions of a Package in Ubuntu
Posted onHow to find all available versions of a package in Ubuntu? To list available versions of a package: apt-cache showpkg <package-name> For example, to check all versions of thunderbird: $ sudo apt-cache showpkg thunderbird Package: thunderbird Versions: 1:31.1.1+build1-0ubuntu0.14.04.1 (/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty-updates_main_binary-amd64_Packages) (/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_trusty-security_main_binary-amd64_Packages) Description Language: File: /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_main_binary-amd64_Packages MD5: 68ed1001b79d708ad48956a0c129114d Description Language: en File: /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en MD5: 68ed1001b79d708ad48956a0c129114d 1:31.0+build1-0ubuntu0.14.04.1 (/var/lib/dpkg/status)
Read more
How to unlock a locked SSH terminal accidentally by ctrl-s on Linux?
Posted onI find I sometimes lock the SSH terminal accidentally by ctrl-s on Linux. How to unlock/unfreeze it? Ctrl-s: lock the SSH terminal. Ctrl-q: unlock the SSH terminal.
How to find and change a user’s UID on Linux?
Posted onHow to find and change a user’s UID on Linux? Find the UID of a user: id -u $USER_NAME Change a user’s UID: # usermod -u $NEW_UID $USER_NAME
How to backup iPhone on Linux?
Posted onHow to backup iPhone on Linux? Not using iTunes or Windows. You can backup iPhone using libimobiledevice. By the time I replied here, it supports up to iOS 7. Please check this post for a tutorial. Actually, the methods introduced can be used on Linux systems with the libimobiledevice installed. Backup iPhone: idevicebackup2 backup ~/iPhoneBackups/
Read more
How to find the history of updated packages by apt-get or aptitude?
Posted onHow to find the history of updated packages by apt-get or aptitude? The history is in log files of dpkg and apt: /var/log/dpkg.log /var/log/apt/history.log /var/log/aptitude To check the recently installed packages: cat /var/log/dpkg.log | grep ” install ” To list history of recently installed packages by apt-get: cat /var/log/apt/history.log | grep ” install ” To
Read more
How to back up emails from the email server?
Posted onofflineimap actually does “synchronization”—if an email is deleted from the server, the email will be deleted from the local Maildir (http://offlineimap.org). This is good for mail sync. However, for backing up emails,one would like to keep the email even if it is deleted from the server. How to achieve this? You may use `getmail` which
Read more