In this post, creating a file-backed virtual block device (VBD) and installing Fedora 20 in the Xen DomU via internet will be introduced. This domain is created on a Fedora 20 Dom0 as introduced in https://www.systutorials.com/installing-xen-on-fedora-20-as-domain-0/. For better performance, you may consider using LVM backed VM. Create file-backed VBD The actual space of VBD will
Read more
Tag: File system
How To Mount Google Drive on Linux
Posted onGoogle Drive is nice cloud storage which provide document editing features. However, it does not yet provide a Linux client. I find a good third party tool that works with Google Drive on Linux very well: google-drive-ocamlfuse. The website of google-drive-ocamlfuse provides easy to follow instructions to install it. In this tutorial, we show a
Read more
How to Flush Linux File System Caches
Posted onWe may drop the file system caches on Linux to free up memory for applications. Kernels 2.6.16 and newer provide a mechanism via the /proc/ to make the kernel drop the page cache and/or inode and dentry caches on command. We can use this mechanism to free up the memory. However, this is a non-destructive
Read more
Setting Up LVM Backed Xen DomU
Posted onLVM volumes as backing for DomU’s file system is an appealing solution to Xen VBD. LVM volumes can dynamically grow/shrink and snapshot. These features make it simple and fast to duplicate DomU and adding storage to DomU. Creating LVM-backed Xen DomU is introduced in this post. Create LVM-backed VBD Suppose we have a LVM volume
Read more
Duplicating LVM Backed Xen DomU
Posted onLVM’s snapshot feature enables us to duplicate an LVM backed Xen DomU in seconds rather than minutes. We no longer need to copy the entire file system image like backing up file backed Xen DomU. We just need to make a snapshot of the current Xen DomU in seconds. When there are changes to the
Read more
Setting up Stable Xen Dom0 with Fedora: Xen 3.4.3 with Xenified Linux Kernel 2.6.32.13 in Fedora 12
Posted onThis is the latest stable and recommended stable Xen Dom0 solution on Fedora 12. No serious bug found till now and we will fix the bugs by ourselves if some appears. It also works on Fedora 14 as well. It should not be hard to use this solution on other versions of Fedora or other
Read more
Set up and Run Linux Xen Dom0 and DomU VMs
Posted onThe Xen solutions including installing and configuring Dom0 and DomU are summarized here. LVM volumes as backing for DomU’s file system is an appealing solution to Xen VBD. LVM volumes can dynamically grow/shrink and snapshot. These features make it simple and fast to duplicate DomU and adding storage to DomU. LVM backed DomU is recommended.
Read more
How to Set Up eCryptFS on Linux – The Manual Way
Posted onHow to set up eCryptFS in Linux will be introduced in this post. We can store encrypted files in one eCryptFS directory, the manual way. The content can be seen only after it is mounted as eCryptFS file system. Otherwise, the users can only see garbled characters in the files. Note that this tutorial will
Read more
Mounting Remote Folder Through SSH
Posted onSSH is a very convenient tool on Linux that can be used to [[port-forwarding-using-ssh-tunnel]], [[proxy-using-ssh-tunnel]] and others besides its basic functions (remote shell). We can also use SSH to mount a remote folder to a local directory. We can use `sshfs` to mount remote folder through SSH tunnel securely over network. Install sshfs Install `sshfs`
Read more
Create and Manage Virtual Machines on Xen
Posted onIn this post, these content are introduced: Create and manage file-backed virtual block device (VBD) for virtual machines on xen. Install Fedora 11 via internet as DomU on top of xen. Manage virtual machines using xm. Create file-backed VBD: The actual space of VBD will be the amount of disk the virtual machine used. And
Read more
Large-scale Data Storage and Processing System in Datacenters
Posted onResearch on Cloud Computing has made big progresses and many excellent large-scale systems have been designed in recent years. I compiled a list of some large-scale data storage and processing systems in datacenters as follows. Storage systems Google File System (GFS): http://research.google.com/archive/gfs.html HDFS implementation: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html Colossus (GFS2): Colossus: Successor to the Google File System (GFS)
Read more
Colossus: Successor to the Google File System (GFS)
Posted onColossus is the successor to the Google File System (GFS) as mentioned in the paper on Spanner at OSDI 2012. Colossus is also used by spanner to store its tablets. The information about Colossus is slim compared with GFS which is published in the paper at SOSP 2003. There is still some information about Colossus
Read more
Hadoop Installation Tutorial (Hadoop 1.x)
Posted onUpdate: If you are new to Hadoop and trying to install one. Please check the newer version: Hadoop Installation Tutorial (Hadoop 2.x). Hadoop mainly consists of two parts: Hadoop MapReduce and HDFS. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce that is initially designed
Read more
Reading List for Distributed Systems and Cloud Computing
Posted onUnderstanding the literature is usually the first step to do research, which is the same for systems research on cloud computing. A reading list may help a lot to those that just start in cloud computing research. Prof. Lin Gu, my PhD supervisor, compiled a reading list for system research on cloud computing. The reading
Read more
mrcc – A Distributed C Compiler System on MapReduce
Posted onThe mrcc project’s homepage is here: mrcc project. Abstract mrcc is an open source compilation system that uses MapReduce to distribute C code compilation across the servers of the cloud computing platform. mrcc is built to use Hadoop by default, but it is easy to port it to other could computing platforms, such as MRlite,
Read more