How to export my Google Chrome password on Linux to a human-readable text file? In newer versions of Chrome, the passwords are stored using the encrypted password storage provided by the system, either Gnome Keyring or KWallet. We need to force Chrome to use a temporary profile folder with unencrypted password storage. Step 1. Connect
Read more
Tag: Storage
How to handle missing blocks and blocks with corrupt replicas in HDFS?
Posted onOne of HDFS cluster’s hdfs dfsadmin -report reports: Under replicated blocks: 139016 Blocks with corrupt replicas: 9 Missing blocks: 0 The “Under replicated blocks” can be re-replicated automatically after some time. How to handle the missing blocks and blocks with corrupt replicas in HDFS? Understanding these blocks A block is “with corrupt replicas” in HDFS
Read more
How to disable the coredump function of systemd on Linux?
Posted onI do not want to have coredump for the developing machine using systemd on Linux? How to disable the coredump function of systemd? You can disable coredump function of systemd by adding this line Storage=none to /etc/systemd/coredump.conf For your reference: Storage= Controls where to store cores. One of “none”, “external”, “journal”, and “both”. When “none”,
Read more
How to check whether a USB disk is connected to a Linux server?
Posted onI can only connect to a Linux server by SSH. How to check whether a USB disk is connected to the Linux server? 2 ways I usuaully use: Method One: Check the print out of dmesg After the USB storage is connected and detected by Linux, it may print some messages containing “[sde]” (the new
Read more
How to disable auto mounting on Linux Mint Cinnamon?
Posted onI prefer Linux Mint (I am using Cinnamon) not automatically mounting a portable storage such as USB disk or iPhone for me and I would like do the mounting manually if I need it. The default behavior of Linux Mint is to automatically mount the disks. How to disable this? The auto mounting on Linux
Read more
How to add a new HDFS NameNode metadata directory to an existing cluster?
Posted onWe have a running HDFS cluster. Currently, the NameNode metadata data directory has only one directory configured in hdfs-site.xml: <property> <name>dfs.namenode.name.dir</name> <value>file:///home/hadoop/hdfs/</value> <description>NameNode directory for namespace and transaction logs storage.</description> </property> We would like to add a new directory for dfs.namenode.name.dir to make replicas of the metadata on a separated disk for higher data reliability.
Read more
how to remove specific directories recursively
Posted onHow to remove .svn directories under hlfs dir recursively as follows. weiwei@weiwei-HP-Compaq-dx6128-MT-PX478AV:~/workshop1/hlfs > find ./ -name “.svn” ./test/build/.svn ./test/.svn ./output/conf/.svn ./output/lib32/.svn ./patches/.svn ./src/include/.svn ./src/include/api/.svn ./src/snapshot/.svn ./src/snapshot/unittest/build/.svn ./src/snapshot/unittest/.svn ./src/utils/.svn ./src/clean/Mapreducer/build/.svn ./src/clean/Mapreducer/.svn ./src/clean/.svn ./src/clean/unittest/.svn ./src/icache/.svn ./src/icache/unittest/.svn ./src/backend/.svn ./src/storage/.svn ./src/cache/.svn ./src/cache/unittest/.svn ./src/clone/.svn ./src/tools/.svn ./src/tools/unittest/.svn ./src/logger/.svn weiwei@weiwei-HP-Compaq-dx6128-MT-PX478AV:~/workshop1/hlfs > find ./ -name “.svn” | xargs rm -rf
How to balance DataNode storage in HDFS?
Posted onAs nodes are added and deleted in a Hadoop cluster. Storage usage across DataNodes may be different. Some DataNodes’ disks are almost used up while some others’ are almost empty. How to balance data across DataNodes in HDFS? Hadoop provides the balancer to redistribute the data. Brief introduction to balancer in Hadoop: balancer. The design
Read more
What’s the difference between Reliability, Durability, and Availability for data storage system?
Posted onSome important concepts in distributed system like Hadoop distributed file system, Google file system and so on. Answer from http://www.quora.com/Whats-the-difference-between-Reliability-Durability-and-Availability-for-data-storage-system The difference between durability and availability is fairly simple. Durability is about what happens when all power goes out everywhere. Has all data been written to stable storage that doesn’t require power (e.g. disk/flash), in
Read more
How to back up emails from the email server?
Posted onofflineimap actually does “synchronization”—if an email is deleted from the server, the email will be deleted from the local Maildir (http://offlineimap.org). This is good for mail sync. However, for backing up emails,one would like to keep the email even if it is deleted from the server. How to achieve this? You may use `getmail` which
Read more
How to use LVM groups with libvirt
Posted onlibvirt uses files for VM images by default under /var/lib/libvirt/images/. LVM is a convenient tool for managing VM images. How to use LVM groups with libvirt? This post discusses the method to add LVM groups as a storage pool for libvirt: http://ailoo.net/2011/02/use-a-lvm-volume-group-with-libvirt/ For a general introduction of the storage management in libvirt, please check: http://libvirt.org/storage.html
HTML form generation from the database and store value into the database
Posted onI have a “t_form” table and a “t_attribute” table. It looks a little bit like below. Form table form_id | form_name | description ———————————– 1 | Laptop | Possible attributes are Model, Screen Size, OS, Cd Player 2 | Mobile | Possible attributes are Model, OS 3 | Tablet | Possible attributes are Model, Screen
Read more
How to find out and change the storage engine of tables in MySQL
Posted onHow to find out and change the storage engine of tables in MySQL databases? Find out the storage engine of a table in a database: SELECT ENGINE FROM information_schema.TABLES WHERE TABLE_SCHEMA = ‘database’ AND TABLE_NAME = ‘table’ Change the storage engine of a table: ALTER TABLE table ENGINE = type type can be innodb or
Read more
MySQL at Facebook
Posted onFacebook uses lots MySQL databases. Any information about how Facebook scales MySQL? Some information on the Web: MySQL at Facebook’s page https://www.facebook.com/MySQLatFacebook?filter=1 A post by Ryan Thiessen, Database Operations at Facebook on Quora: http://www.quora.com/Facebook-Engineering/How-does-Facebook-structure-MySQL-so-that-it-is-robust-and-scalable And more: http://mashable.com/2011/12/15/facebook-timeline-mysql/ http://gigaom.com/2011/12/06/facebook-shares-some-secrets-on-making-mysql-scale/ http://www.wired.com/wiredenterprise/2011/12/facebook-timeline-anatomy “A lot of people are surprised that for this shiny new thing for Facebook, we’re using
Read more
SQL layers on NoSQL databases
Posted onWhat are the SQL layer solution over NoSQL databases such as key/value stores? Phoenix: A SQL layer on HBase: https://github.com/forcedotcom/phoenix They also show some performance results: https://github.com/forcedotcom/phoenix/wiki/Performance F1 – The Fault-Tolerant Distributed RDBMS Supporting Google’s Ad Business: http://research.google.com/pubs/pub38125.html With F1, we have built a novel hybrid system that combines the scalability, fault tolerance, transparent sharding,
Read more
ANSI C Grammar and ANSI C Grammar Specification in YACC
Posted onANSI C is a standardized version of the C programming language that was introduced in 1989 by the American National Standards Institute (ANSI). The ANSI C standard defines the syntax and semantics of the C language, including its grammar. In this post, we will explore the ANSI C grammar in more detail. The ANSI C
Read more
Managing LVM
Posted onAny tutorials or tips on how to manage LVM? LVM provides a flexible and easy way to management storage disks on Linux. It also provides a set of tools for management. Here are some tutorials and references for managing LVM: LVM man pages:Â https://www.systutorials.com/docs/linux/man/8-lvm/#lbAL Manage Disk Volume in Linux:Â http://www.hongkedavid.com/blog/disk_volume.html LVM with Xen:Â https://www.systutorials.com/b/linux/tag/lvm/ Remove missing phyiscal volumes
Read more
Where Does Evolution Save Its Data and Configuration Files on Linux?
Posted onEvolution is a great personal information management tool that provides Email, address book and calendar tools. Evolution provides many enterprise friendly feature such as native support to Microsoft Exchange connectivity for Emails, address books and calendars. Evolution uses various ways including plain files and dconf configuration systems. This post will give an introduction to the
Read more
How to Upload Large Files to Amazon S3 with AWS CLI
Posted onAmazon S3 is a widely used public cloud storage system. S3 allows an object/file to be up to 5TB which is enough for most applications. The AWS Management Console provides a Web-based interface for users to upload and manage files in S3 buckets. However, uploading a large files that is 100s of GB is not
Read more
Which Checksum Tool on Linux is Faster?
Posted onIt is common practice to calculate the checksums for files to check its integrity. For large files, the checksum computation is slow. Now I am wondering why it is so slow and whether choosing another tool will be better. In this post, I try three common tools md5sum, sha1sum and crc32 to compute checksums on
Read more