{"id":5725,"date":"2013-10-30T17:00:30","date_gmt":"2013-10-30T16:00:30","guid":{"rendered":"https:\/\/ingmarverheij.com\/?p=5725"},"modified":"2013-11-08T09:15:26","modified_gmt":"2013-11-08T08:15:26","slug":"devsda1-has-gone-255-days-without-being-checked-check-forced","status":"publish","type":"post","link":"https:\/\/ingmarverheij.com\/en\/devsda1-has-gone-255-days-without-being-checked-check-forced\/","title":{"rendered":"\/dev\/sda1 has gone 255 days without being checked, check forced"},"content":{"rendered":"<p><a href=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/UNEXPECTED-INCONSISTENCY.png\"><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; float: right; display: inline; background-image: none;\" title=\"UNEXPECTED INCONSISTENCY; run fsck MANUALLY\" alt=\"\/dev\/sda1 has gone 255 days without being checked, check forced\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/UNEXPECTED-INCONSISTENCY_thumb.png\" width=\"154\" height=\"116\" align=\"right\" border=\"0\" \/><\/a>On initial boot of a <strong>freshly deployed <\/strong>(Ubuntu) Linux appliance on <strong>VMware vSphere 5<\/strong> the machine stopped working because of an inconsistent disk. According to the machine <strong>255 days<\/strong> where passed without the disk being <strong>checked<\/strong>.\u00a0 After a file system <strong>check <\/strong>is <strong>forced <\/strong>the machine stops with an error <em>UNEXPECTED INCONSISTENCY<\/em><strong> <\/strong>followed by <em>fsck \/ [5001] terminated with status 4<\/em>.<\/p>\n<p><!--more--><\/p>\n<h1>\/dev\/sda1 has gone x days without being checked<\/h1>\n<p>The first message was very strange since the machine was freshly deployed from an <a href=\"https:\/\/en.wikipedia.org\/wiki\/Open_Virtualization_Format\" target=\"_blank\">OVF<\/a>. According to the vendor this issue occurs when the date and time on the Hypervisor Host is wrong. After correcting the date and time of the hypervisor to the current date and time this problem should be resolved.<\/p>\n<p>I <strong>verified<\/strong> the date and time of the ESX host, virtual machine and my laptop; <strong>they all matched<\/strong> (although the ESX host and virtual machine show the UTC time and I run in DST (+1). Since this problem occurred on October 29th 2013, I tried changing the time to the <strong>past<\/strong> hoping to <strong>trick<\/strong> the machine. During the attempts I found the <em>Superblock last mount time<\/em> of the devices:<\/p>\n<ul>\n<li><strong>\/dev\/sda1<\/strong> \u2013 Fri Feb 15 10:53:23 2013<\/li>\n<li><strong>\/dev\/sda6 <\/strong>\u2013 Fri Aug 16 18:32:41 2013<\/li>\n<\/ul>\n<p>The results of my test where unsuccessful, but it\u2019s worth sharing anyway<\/p>\n<p><strong>Attempt #1<\/strong>\u00a0 &#8211; January 1st 2013<\/p>\n<ul>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error.png\" width=\"10\" height=\"11\" border=\"0\" \/>\/dev\/sda1<\/strong> \u2013 Superblock last mount time (Fri Feb 15 10:53:34 2013, now = Tue Jan 1 14:49:08 2013) is in the future<\/li>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error1.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda6 <\/strong>\u2013 Superblock last mount time (Fri Aug 16 19:32:31 2013, now = Tue Jan 1 14:49:08 2013) is in the future<\/li>\n<\/ul>\n<p><strong>Attempt #2 <\/strong>&#8211; May 15th\u00a0 2013<\/p>\n<ul>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Checkmark.png\" width=\"10\" height=\"11\" border=\"0\" \/>\/dev\/sda1<\/strong> \u2013 Clean<\/li>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error2.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda6 <\/strong>\u2013 Superblock last mount time (Fri Aug 16 19:32:31 2013, now = Tue Jan 1 14:49:08 2013) is in the future<\/li>\n<\/ul>\n<p><strong>Attempt #3 \u2013 <\/strong>August 8th\u00a0 2013<\/p>\n<ul>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Checkmark1.png\" width=\"10\" height=\"11\" border=\"0\" \/>\/dev\/sda1<\/strong> \u2013 Clean<\/li>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error3.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda6 <\/strong>\u2013 Superblock last mount time (Fri Aug 16 19:32:31 2013, now = Thu Aug 18 15:07:31 2013) is in the future<\/li>\n<\/ul>\n<p><strong>Attempt #4 \u2013 <\/strong>August 15th 2013<\/p>\n<ul>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error4.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda1<\/strong> \u2013 \/dev\/sda1 gas gone 180 days without being checked, check forced<\/li>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error5.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda6 <\/strong>\u2013 Superblock last mount time (Fri Aug 16 19:32:31 2013, now = Thu Aug 15 18:33:05 2013) is in the future<\/li>\n<\/ul>\n<p><strong>Attempt #5 \u2013 <\/strong>August 16th 2013<\/p>\n<ul>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error6.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda1<\/strong> \u2013 \/dev\/sda1 gas gone 182 days without being checked, check forced<\/li>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error7.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda6 <\/strong>\u2013 \/dev\/sda6 gas gone 182 days without being checked, check forced<\/li>\n<\/ul>\n<p><strong>Attempt #6 \u2013 <\/strong>December 12th 2013<\/p>\n<ul>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error8.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda1<\/strong> \u2013 \/dev\/sda1 gas gone 299 days without being checked, check forced<\/li>\n<li><strong><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; margin: 0px 5px 0px 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; display: inline; background-image: none;\" title=\"\" alt=\"\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Error9.png\" width=\"10\" height=\"11\" border=\"0\" \/><\/strong><strong>\/dev\/sda6 <\/strong>\u2013 \/dev\/sda6 gas gone 299 days without being checked, check forced<\/li>\n<\/ul>\n<p><a href=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Attemp-August-15th-2013.png\"><img loading=\"lazy\" decoding=\"async\" style=\"border-width: 0px; padding-top: 0px; padding-right: 0px; padding-left: 0px; margin-right: auto; margin-left: auto; float: none; display: block; background-image: none;\" title=\"Attemp - August 15th 2013\" alt=\"Attemp - August 15th 2013\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/Attemp-August-15th-2013_thumb.png\" width=\"354\" height=\"131\" border=\"0\" \/><\/a><\/p>\n<p>As you can see I tried a lot (while not all of them made sense, I agree) but I could not find the sweet spot. Time for the next approach!<\/p>\n<p>&nbsp;<\/p>\n<h1>fsck \u2013 file system check<\/h1>\n<p>The machine <strong>clearly<\/strong> indicated <strong>two<\/strong> problems:<\/p>\n<ol>\n<li>The last file system check was executed 255 days ago<\/li>\n<li>There are errors with the consistency of the disk (filesystem has errors)<\/li>\n<\/ol>\n<p>Since I found no other solution I addressed both problems at the same time: <strong>run a file system check<\/strong>.<\/p>\n<p>Since the machine halted I was unable to run any command, including <a href=\"https:\/\/www.freebsd.org\/cgi\/man.cgi?query=fsck&amp;sektion=8\" target=\"_blank\">fsck<\/a>,. So I mounted a <a href=\"https:\/\/en.wikipedia.org\/wiki\/List_of_live_CDs\" target=\"_blank\">Linux Live CD<\/a> (<a href=\"https:\/\/fedoraproject.org\/wiki\/FedoraLiveCD\" target=\"_blank\">FedoraLiveCD<\/a> in my case, but you could also use the <a href=\"https:\/\/help.ubuntu.com\/community\/LiveCD\" target=\"_blank\">Ubuntu LiveCD<\/a>) and ensured it booted from CD (change the boot order). After starting a shell (Terminal) I ran the <strong><a href=\"https:\/\/linux.about.com\/od\/commands\/l\/blcmdl1_su.htm\" target=\"_blank\">su command<\/a><\/strong> to elevate my permissions.<\/p>\n<pre lang=\"bash\">su<\/pre>\n<p>Next I ran the <a href=\"https:\/\/www.freebsd.org\/cgi\/man.cgi?query=fsck&amp;sektion=8\" target=\"_blank\">fsck<\/a> command on the <em>\/dev\/sda1<\/em> device with the <strong>\u2013f <\/strong>argument\u00a0 to <strong>force checking <\/strong>of the file system and the <strong>\u2013y<\/strong> argument to answer <strong>Yes <\/strong>on all questions.<\/p>\n<pre lang=\"bash\">fsck \/dev\/sda1 -f \u2013y<\/pre>\n<p><a href=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/fsck-dev-sda1-f-y.png\"><img loading=\"lazy\" decoding=\"async\" style=\"border: 0px currentColor; padding-top: 0px; padding-right: 0px; padding-left: 0px; margin-right: auto; margin-left: auto; float: none; display: block; background-image: none;\" title=\"fsck \/dev\/sda1 -f -y\" alt=\"fsck \/dev\/sda1 -f -y\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/fsck-dev-sda1-f-y_thumb.png\" width=\"354\" height=\"116\" border=\"0\" \/><\/a><\/p>\n<p>And the same command but for the the <em>\/dev\/sda6<\/em>device,.<\/p>\n<pre lang=\"bash\">fsck \/dev\/sda6 -f -y<\/pre>\n<p><a href=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/fsck-dev-sda6-f-y.png\"><img loading=\"lazy\" decoding=\"async\" style=\"border: 0px currentColor; padding-top: 0px; padding-right: 0px; padding-left: 0px; margin-right: auto; margin-left: auto; float: none; display: block; background-image: none;\" title=\"fsck \/dev\/sda6 -f -y\" alt=\"fsck \/dev\/sda6 -f -y\" src=\"https:\/\/ingmarverheij.com\/wp-content\/uploads\/2013\/10\/fsck-dev-sda6-f-y_thumb.png\" width=\"354\" height=\"205\" border=\"0\" \/><\/a><\/p>\n<h1>Success<\/h1>\n<p>Sure enough (after unmounting the ISO) the machine now boots succesfull.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>On initial boot of a freshly deployed (Ubuntu) Linux appliance on VMware vSphere 5 the machine stopped working because of an inconsistent disk. According to the machine 255 days where passed without the disk being checked.\u00a0 After a file system check is forced the machine stops with an error UNEXPECTED INCONSISTENCY followed by fsck \/ [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-container-style":"default","site-container-layout":"default","site-sidebar-layout":"default","disable-article-header":"default","disable-site-header":"default","disable-site-footer":"default","disable-content-area-spacing":"default","footnotes":""},"categories":[592],"tags":[606,604,605,76],"class_list":["post-5725","post","type-post","status-publish","format-standard","hentry","category-vmware-2","tag-fsck","tag-linux","tag-ubuntu","tag-vmware"],"_links":{"self":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts\/5725","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/comments?post=5725"}],"version-history":[{"count":3,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts\/5725\/revisions"}],"predecessor-version":[{"id":5759,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts\/5725\/revisions\/5759"}],"wp:attachment":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/media?parent=5725"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/categories?post=5725"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/tags?post=5725"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}