In this post, creating a file-backed virtual block device (VBD) and installing Fedora 20 in the Xen DomU via internet will be introduced. This domain is created on a Fedora 20 Dom0 as introduced in https://www.systutorials.com/installing-xen-on-fedora-20-as-domain-0/. For better performance, you may consider using LVM backed VM. Create file-backed VBD The actual space of VBD will…
Google Drive is nice cloud storage which provide document editing features. However, it does not yet provide a Linux client. I find a good third party tool that works with Google Drive on Linux very well: google-drive-ocamlfuse. The website of google-drive-ocamlfuse provides easy to follow instructions to install it. In this tutorial, we show a…
We may drop the file system caches on Linux to free up memory for applications. Kernels 2.6.16 and newer provide a mechanism via the /proc/ to make the kernel drop the page cache and/or inode and dentry caches on command. We can use this mechanism to free up the memory. However, this is a non-destructive…
LVM’s snapshot feature enables us to duplicate an LVM backed Xen DomU in seconds rather than minutes. We no longer need to copy the entire file system image like backing up file backed Xen DomU. We just need to make a snapshot of the current Xen DomU in seconds. When there are changes to the…
LVM volumes as backing for DomU’s file system is an appealing solution to Xen VBD. LVM volumes can dynamically grow/shrink and snapshot. These features make it simple and fast to duplicate DomU and adding storage to DomU. Creating LVM-backed Xen DomU is introduced in this post. Create LVM-backed VBD Suppose we have a LVM volume…
This is the latest stable and recommended stable Xen Dom0 solution on Fedora 12. No serious bug found till now and we will fix the bugs by ourselves if some appears. It also works on Fedora 14 as well. It should not be hard to use this solution on other versions of Fedora or other…
The Xen solutions including installing and configuring Dom0 and DomU are summarized here. LVM volumes as backing for DomU’s file system is an appealing solution to Xen VBD. LVM volumes can dynamically grow/shrink and snapshot. These features make it simple and fast to duplicate DomU and adding storage to DomU. LVM backed DomU is recommended….
How to set up eCryptFS in Linux will be introduced in this post. We can store encrypted files in one eCryptFS directory, the manual way. The content can be seen only after it is mounted as eCryptFS file system. Otherwise, the users can only see garbled characters in the files. Note that this tutorial will…
SSH is a very convenient tool on Linux that can be used to [[port-forwarding-using-ssh-tunnel]], [[proxy-using-ssh-tunnel]] and others besides its basic functions (remote shell). We can also use SSH to mount a remote folder to a local directory. We can use `sshfs` to mount remote folder through SSH tunnel securely over network. Install sshfs Install `sshfs`…
In this post, these content are introduced: Create and manage file-backed virtual block device (VBD) for virtual machines on xen. Install Fedora 11 via internet as DomU on top of xen. Manage virtual machines using xm. Create file-backed VBD: The actual space of VBD will be the amount of disk the virtual machine used. And…
Research on Cloud Computing has made big progresses and many excellent large-scale systems have been designed in recent years. I compiled a list of some large-scale data storage and processing systems in datacenters as follows. Storage systems Google File System (GFS): http://research.google.com/archive/gfs.html HDFS implementation: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html Colossus (GFS2): Colossus: Successor to the Google File System (GFS)…
Colossus is the successor to the Google File System (GFS) as mentioned in the paper on Spanner at OSDI 2012. Colossus is also used by spanner to store its tablets. The information about Colossus is slim compared with GFS which is published in the paper at SOSP 2003. There is still some information about Colossus…
Update: If you are new to Hadoop and trying to install one. Please check the newer version: Hadoop Installation Tutorial (Hadoop 2.x). Hadoop mainly consists of two parts: Hadoop MapReduce and HDFS. Hadoop MapReduce is a programming model and software framework for writing applications, which is an open-source variant of MapReduce that is initially designed…
Understanding the literature is usually the first step to do research, which is the same for systems research on cloud computing. A reading list may help a lot to those that just start in cloud computing research. Prof. Lin Gu, my PhD supervisor, compiled a reading list for system research on cloud computing. The reading…
The mrcc project’s homepage is here: mrcc project. Abstract mrcc is an open source compilation system that uses MapReduce to distribute C code compilation across the servers of the cloud computing platform. mrcc is built to use Hadoop by default, but it is easy to port it to other could computing platforms, such as MRlite,…