Replace Disks in a Failed RAID1 and Grow it Afterwards

One week ago, my Linux NAS decided to spam me with mails, while I was sitting in a bar. Nothing is more disturbing, while having a nice chat and drink a beer, than your home server screaming about a failed RAID. I have to admit, I knew the RAID will fail soon, since one of the HDDs already was marked bad by SMART for some weeks…

Anyway, I shut down the server remotely and ordered immediately two new Seagate 4 TB NAS Drives. They arrived soon, but then my struggle begun. I really had no idea how to replace the HDDs in the RAID array and then grow the RAID to the new size of the HDDs, since my old drives where only 2 TB.

After some time of Google research I was aware of the steps to perform the procedure and it’s fairly easy!

(Be aware that identifier in your system might be different! Perform this steps at your own risk and make a backup before you perform anything on your system!)

  1. Shutdown the server and replace the failed HDD with the first new HDD.
  2. Now its time to create a new partition on the drive. Since my partition will be greater than 2 TB, I need to use GPT layout on the drive. This can be archived with the parted command.

    Now I have the GPT in place and can add a partition. I’ll use the whole drive as one big partition and use ext4:
  3. Add the new HDD to the RAID array and wait for the re-sync. This can take a long time, in my case 3.5 h.
  4. After the re-sync, remove the last old HDD with the following commands:
  5. Now shutdown the system again and replace the second old HDD with a new one.
  6. Repeat the steps 2 and 3 on the new drive.
  7. Now it’s time to grow the array. Let’s have a look first:

    Ok, looks great. Notice the line 4 where the block number is displayed. 1953382208 1024K blocks equal to 2 TB or 1.82 TiB. Now let’s gow the array with mdadm:
  8. So, now the virtual disk is bigger, but the partition is still 2 TB. So we have to grow this as well:
  9. And we are done!

I repeated some steps in a virtual machine for this blog post, so if you find some mismatches in the sizes etc. its most likely from that. Feel free to drop a comment for errata requests! 🙂

 

Raspberry PI as WiFi to Ethernet Bridge

Recently my cable modem refused to connect to my provider, stating “Connection Refused”. The provider (UPC Austria) stated, that they have to come to my place and make some measurements. But this will be in one week…. WTF?? One week without internet? No way!

Fortunately, my provider offers a service, that every router provides two networks, a private one for the single customer, and a free network for all customers of UPC. This “UPC Wi-Free” was also available from some neighbours of mine.

It would be easy to connect all of my PCs to the network, but this is not enough for me. I needed a better solution. Also the PC of my girlfriend refused to connect to this network, my big PC has no WiFi and the server for sure not. Also the chromecast and the firestick…. no I will not configure all devices to configure them later again!

My previous setup included the router from my provider connected to my own router, a Fritz Box. All my devices are then behind the Fritz Box. So the only thing I had to change was to replace the UPC router with my Raspberry PI.

My starting point was the following:

  • Raspberry PI 2B
  • Alfa AWUS036H Wifi Adapter
  • a network cable 🙂

First I powered up the Raspberry and was curious what I had done with this firmware before. Apparently, I used it before for some photographing stuff… 🙂 Anyway, first the Debian needed to be updated from 7 (wheezy) to 8 (jessie).

First we make sure the current system is up to date:

Then we edit /etc/apt/sources.list and replace wheezy with jessie

Then the upgrade:

This took a while, the raspi is not a fast gaming machine… After the upgrade, I configured the WiFi adapter to work with the UPC Wi-Free. For this the wpa_supplicant and the the network interfaces have to be configured:

The following content belongs to /etc/wpa_supplicant/wpa_supplicant.conf:

And this belongs to /etc/network/interfaces:

You can use different IPs for sure and also you can specify any other DNS than Googles 8.8.8.8. Now we restart the network service and test the connection:

The Ethernet interface need a static IP in order to serve DHCP. For this we need also a DHCP server installed on the raspi. We achieve this with the ISC DHCP Server.

We need then to configure the server with a really basic configuration. The configuration is done by editing the file /etc/dhcp/dhcpd.conf.

Basically the set here some lease times for the server, with authoritative we tell the server that he will server this range alone. Later we define a small subnet. The “option routers” is important, because here the DHCP server will tell clients that he will also serve requests later as the gateway.

Now we can start the server:

Now we can connect the router and we will see that the raspi serves with an IP address. Now we have to connect both interfaces. In my case the interfaces have the names eth0 and wlan0. We use iptables for this:

This is only temporary so we have to make the rules persist. The easy way here is with the package iptables-persistent:

During the installation you will be asked to save the current rules, answer with yes and the installer will save the rules to /etc/iptables/rules.v4:

Now you have to activate also the ipv4 forwarding by executing the following line:

And you are done!

Update:

THX to the users u/ZoLustIkErNogWelEen and u/Q3_benji who pointed out I forgot to write down the steps with the sources.list and the activation of the ipV4 forwarding!

Update2:

THX to Rob who showed me a mistake in the post!

Had no money for a slider. Build my own…

All these years I wanted to buy myself a slider for time lapses. All these years they where to expensive for me. The good ones like the Stage One from http://www.dynamicperception.com/ will cost you, depending on the configuration up to $1,250.00. Without shipping to Europe. Even the small ones like the Rhino Slider EVO Carbon are at least $500, again without shipping. So I decided to build my own.

I ordered some aluminum pipes, a aluminum plate, four teflon bearings, two aluminum blocks and used a 3D printer for some other parts. After some time of sticking, milling, drilling and printing, this is the outcome.

Next, I will order a Astro devices with the linear mount.

Night Fox – Night Sky Photography Shutter Speed Calculator Android App

EDIT: Dropbox recently closed the public folder system where I store my camera database. Therefor the app was crashing while updating the database. I fixed this issue and the new version 1.3.0 is going live now.

Due to the overall great response and the high number of mails from users of my Shutter Speed Calculator i’m glad to announce here: Night Fox – Shutter Speed Calculator for Android!

The App features a simple interface to input your camera, your focal length and the tolerance in pixels. After that you can simply hit the calculate button and read the suggested speeds.

If you cannot find your camera in the list, simply write me an email or leave a comment. I can add cameras live and you just have to hit the “Download Camera File” Button to load the newest definition data. If you enjoy the app, please leave a comment and help me improve the interface and the user experience!

You can get the app for free from here:
Get it on Google Play

 

If you want to learn more about how it works, check out my two other posts:
https://www.tl-photography.at/stars/shutter-speed-calculator-for-night-sky-photographing/
https://www.tl-photography.at/stars/night-sky-photography-shutter-speed-calculator/

Frequency Measurement of Vienna

Update2: Because of the massive amount of traffic I received I had to disable the live measurement and block some IPs who where causing my server to slow down…

Update: Here are the measurements from the 20. of March, the day of the solar eclipse. And as you can see the grid operators were well prepared. The frequency was actually less fluctuating than the days before.

Frequency Measurements of the 18., 19. and 20. of March 2015

Today I decided to take again online my grid frequency measurement unit. In some days there will be a solar eclipse in Europe and maybe we will notice a slight fluctuation of the frequency during the eclipse. If it will be a sunny day, the shadow of the moon will race over the ground and will switch of the PV plants in Europe. The installed PV generation in Europe is according to this LINK 81,5 GW. But the overall installed power is more that 1300 GW.  So a black out will not happen.

But so far, have fun:

Background:

The grid frequency is a very important measurement factor of the grid. In the electric grid, the balance between generation and demand, due to the lack of a high number of storage, have to be equal at every time. The frequency is a performance indicator in which state this balance is. Lower than 50 Hz the generation is less than the demand, and vice versa. The frequency is therefore used to adapt the power output of power plants to the demand.

Parabola Theme and reCAPTCHA

recaptcha wordpress loginToday I installed reCAPTCHA to get rid of spam. The installation was really simple and I noticed really fast that my IP Blacklist plugin had a little time to relax. But I noticed that the login page look a bit odd. I googled a bit but didn’t find anybody whit the same problem. Also nobody answered in the google group about my question. So I researched myself and found the wordpress tutorial to change your login page.

I added the following to my functions.php, just to increase the size of the login page to 400px, and voila, better.

I hope this can help you with your problems… 🙂

Best Time to Upload and Get Popular on 500px

UPDATE 3: recently 500px changed the website and you cannot see the number of likes anymore…

UPDATE 2: Now i’m continuing the test, but this time with an another account and a new picture. I’ll now wait the whole 24 hours period of 500px and upload only once a day.

Test Uploads, Score of the Picture according to 500px Pulse and the absolute Views/Likes/Favorites 
Date/Time 8 am 10 am 12 am 2 pm 4 pm 6 pm 8 pm 10 pm 12 pm 2 am
Monday  88.2
166/-/22
 88.9
170/27/4
88
199/27/9
 90.5
197/32/8
 85.1
33/19/3
Tuesday 88.3
213/22/10
90.6
219/31/7
63,9
42/5/2
Wednesday 91.3
363/34/9
89.7
146/25/11
56,6
12/4/1
Thursday  87.7
393/23/6
 87.1
123/21/8
iP  93.0
530/43/11
Friday
Saturday
Sunday 60.1
28/5/1
 85.0
97/18/4

UPDATE: I just stopped this test, because I realized, that the users rating is influencing the tests. When I was uploading normally my pictures before, the first Like was worth about 27 Pulse, the second something around 15 and so on. Whit my test is was upload 5-6 times per day the same picture. At the last day, a Like was worth 10 Pulse! With this, my test will deliver really bad results and completely destroy me repetition on 500px. I decided to slow down this test and upload the picture not so often anymore. Maybe I will use the whole 24 hours span of 500px to get the real rating.

I’m not completely sure how the algorithm works, but according to my findings, a click on Like is not worth 27 Pulse at all time and all users. Below, the rating of three pictures can be seen. (Sorry for German… 🙂 ) All of them have the same Likes and Favorites, but different Rating. All of them are from the “New” section at the same 10 minutes. And all of them had no comments. The first one is mine.  It seems that the view to like ratio does not influence the rating that much, since the third picture hat more views but a higher score. Whats left over, is the users rating. Correct me if i’m wrong!

Rating

Since some time I’m participate on 500px.com, to share my photos and get some feedback on the composition and processing etc. My choice felt to 500px.com because I don’t want to create an Yahoo account, I simply didn’t thought about Tumblr and I hate the iPhone majority on Instagram (yes, I have a rooted Android :P). Anyway, 500px offers with the rating system an interesting way of measure the, let’s say, “Beauty” of your pictures, compositions and processing. Also you can have the pictures still with a CC license (maybe the others have this as well, donno…). After I uploaded some photos I got exited. I was spending nights with watching other photos, like them and post some comments. On the other side my pictures get a relatively good rank and some nice comments. But, for high quality critics, go somewhere else. I had some nice rating over 90 and 95 also. 500px has a nice rating system called Pulse. If you want to sell pictures, a high rating, Pulse, is recommended.

Still my best one: The Battle of the Nations Monument in Leipzig.

When the days got longer, I had some time at the golden hour and blue hour to take some pictures.  I also had a new lens and I was still exited with my new 6D. I started photographing at 5 pm and ended at 9 pm or later. After that processing the pictures was not in my mood and I left that for the next day. So i stood up, made some processing and uploaded the pictures, let’s say, between 9:00 and 9:30 (CET) in the morning, three, four days in a row. I was really surprised that the rating where just around 70-80, sometimes less. Could it be that I’m, not like I thought a little bit good, really bad at photographing?

Photograph DC Tower Vienna by Thomas on 500pxBut I was going curious. I noticed also that the count of viewers where not that high like before. OK, no views, no likes, no rating. Even the best pictures of 500px have normally a 1-to-10 rating for “likes” to views and even less with “favorite“. So I made a simply test with one of my pictures. The photo on the right is the DC Tower in Vienna. Not the greatest picture, I know but a 80 worth for sure. The first time i uploaded this picture at around 9-10 AM CET, the pulse stood still after some hours, at around 70. The second time, I uploaded at 8 PM CET. And voilà, the pulse reached 90.7. Also the views where going from some under 100 to over 400. So there are clearly some good and bad times for uploading photos to 500px.

I decided to make a test with one of my newest pictures from Vienna. Also not a very good picture, but clearly worth a 80 or 85. I will upload this picture the next weeks to different time and will not interact with 500px at all, only to see the rating and the views after 2 hours and the upload to a different time again. I started today again at 9:30 AM CET. I’ll try to update this post recently to the latest results as soon as possible.

 

Test Uploads, Score of the Picture according to 500px Pulse and the absolute views
Date/Time 8 am 10 am 12 am 2 pm 4 pm 6 pm 8 pm 10 pm 12 pm 2 am
Monday 67.0
Tuesday  84.9 86.9
Wednesday 82.1  75.9 50.2  85.2 56.4 56,6/26
Thursday 65.6/34 68,6/24  69.8/35  58,5/21
Friday  39.6/8 iP
Saturday
Sunday

 

Night Sky Photography Shutter Speed Calculator

UPDATE: I’m glad to announce that I finally found the time to release the Night Fox Android App for my Shutter Speed Calculator!

Get it on Google Play

This is the shutter speed calculator for night sky photographing. Basically you just insert the data of crop factor, the megapixel you want to archive or your camera has, the focal length and the tolerance of pixels you can accept. The description of all these different factors is below. The description of the whole process can be found here.























Crop Factor
The crop factor is depending on your camera. It is influencing the field of view (FOV) of your camera, like the focal length also does. This calculator includes the factors 1 (3:2), 1.5 (3:2), 1.6 (3:2) and now also 2.0 (4:3). (Thx to Livio for the comment)
See also http://en.wikipedia.org/wiki/Crop_factor and http://en.wikipedia.org/wiki/Aspect_ratio_(image).

Megapixel
This is the amount of megapixels you want to achieve. It can be the maximum of your camera or also a lower Value. I included this value, because somebody may what to have the pictures not in full resolution, but a maximum of exposure time. With a smaller resolution I’m recommending to reduce the pixel tolerance as well.

Focal Length
The focal length is very important for the field of view and therefore how fast the stars are moving. The rule of thumb is “the lower the better”. But be aware of the speed of your lens. This influences the needed exposure time as well.

Pixel Tolerance
Depending on what you can live with, you can adjust this value. Basically it describes the tolerance of how many pixels a star can “move”. In some cases 20 pixels are OK, but for having really sharp stars, I recommend 10 pixel.

The Milky Way Exposure Calculator can be found here and as well as the other articles of http://www.lonelyspeck.com/.

WP Parabola Theme and Google Structured Data

While setting up my Google Webmaster Tools I ran into the problem that WordPress in combination with the nice Parabola Theme is not 100% conformed the Google expectations. I saw some errors on the page and tried to figure out what exactly triggers this error. On the overview page I found these entries.

google webmaster tools failure

 

The crawler of Google searches for the hatom information on the page. For my page it finds most of the tags, like the author, the title, the content and also the publish date. But it searches also specific for the field “updated”. The failure occures more often on category sites, because there are more entries, and only once on the simple post.

After some research I found a solution by adding  some simple lines to some files in the theme. But because you still want to update your theme, the best way to edit your theme is by adding a child to the normal theme. This is basically done by creating a new folder under /wp-content/themes/ with the name %theme-name%-child. In my case this was “parabola-child”. After creating a child folder, a file named sytle.css has to be created in the folder. The basic setup is in my case:

 The most important line is  Template: parabola. This tells WordPress that this is a child of the Parabola theme. All steps can be found here. WordPress will now load all files from the child folder and override the corresponding files of the main theme. Basically you can copy a file like header.php to the child theme and edit it. The rest of the files will be loaded from the main theme. In our case I searched the content.php and found a link to the parabola_posted_on() in includes/theme-loop.php function. This function provides the meta information of the post and can also be used to write the additional tags to the page. Unfortunately, after i copied and edited my file, WordPress was still loading the main file. This seems to be a bug or i did simply something wrong, ten times or so… Anyway, I decided to change in this case the content.php file. I copied the  snipped from here and added the “hidden=true” to prevent that information from rendering to the screen.

After this modification, the category pages will have the hidden information and Google will crawl these pages without errors. But the single post will still have this failure. So again, after some searching, I figured out that this is related to the file single.php. So I added the same two lines there.

 google_webmaster_tools_failure_detailIt seems that the theme also deliverers wrong data to the crawler when showing single posts. In the picture on the side,it is visible that it writes for the bookmark the date and not the name of the post. But i was fine with the result until now. Maybe i’ll fix this later or write a bug report to the programmer.