How to get logs of a specific time range on Linux?

Posted on In QA

The logs I am processing is Hadoop log (log4j). It is in format like:

2014-09-20 21:55:11,855 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated user map size: 36
2014-09-20 21:55:11,863 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Updated group map size: 55
2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Update cache now
2014-09-20 22:10:11,907 INFO org.apache.hadoop.nfs.nfs3.IdUserGroup: Not doing static UID/GID mapping because '/etc/nfs.map' does not exist.

Now, I want to get all the logs with a specific time range, e.g. last 4 hours. How to achieve this?

It should be with command line tools since it is in an automatic routine which is invoked by crond every 4 hours.

You can use date to generate filtering rules to filter out the logs in a specific range:

# grep out latest log
echo "" >$tmplog
for ((i=4; i>=1; i--)); do
    grep "^$(date -d -${i}hour +'%Y-%m-%d %H')" $log >> $tmplog
done

Eric Ma

Eric is a systems guy. Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

Leave a Reply

Your email address will not be published. Required fields are marked *