Here’s A Job That’s Still Done Better By Humans

Where I grew up, there were five radio stations: Country, Country, Classical, Talk, and Rock. My folks liked older country music (and the hosts on that station were probably the best in the area), so I listened to a lot of it. One of the songs I liked because it was so interesting was “The Auctioneer” by Leroy Van Dyke. I got a tape recording of it one day when it was on and tried to emulate his way of talking. Never quite got it, but I could understand what he was saying.

Turns out, the most modern software managed by the largest tech titans on the planet can’t even get that far. I stumbled across a video of Van Dyke performing his song on a YouTube channel called Country Road TV. If you haven’t heard the song before, here he is singing it (he first recorded it back in 1956!):

I went to skip backwards to listen to part of the song again but missed and accidentally turned on captioning. The results were… well, see for yourself:

Keep in mind that the song is about a guy who wants to learn how to talk like an old-school auctioneer; I believe he sells a goat in one part. Nothing about monocles or tables…

And the final run-off at the end of the song is even better:

It’s like the whole “There’s a Bathroom on the Right” thing, but with computers. I am pretty sure that BitDefender didn’t exist in 1956, but who knows – maybe Mr. Van Dyke updated the lyrics.

While most people today probably aren’t looking up captions for 60+ year old novelty country music, issues like this are a real problem for people who rely on captioning for important and accurate information… and it seems the technology still has a lot of room to improve.

Removing IR Filter From ESP32-CAM, Part II

**UPDATE (2020-05-08): I did a little bit of math and realized I was only running the IR illuminator at about a quarter of a watt. After a bit more poking around and squinting at part numbers, I’m pretty sure it can run at almost 10x the power. I will look into this further sometime soon.**

Shortly after my previous post on removing the IR-cut filter from the OV2640 camera that comes with the ESP32-CAM, I put the ESP in a safe place… so safe that I lost it.

I found it again a couple of days ago and finally got to tinkering with it this evening, with the plan to see how well it worked in the dark.

I changed the following settings from the defaults for all of the pictures taken with the ESP in this post:
– Frame size: 1024×768 (XGA)
– Gain ceiling set to 3
– Special effect set to 2 (greyscale)

Without the filter, the colours were all wrong and it was a little harder to focus. This is normal but a little annoying, so I set the image to greyscale. Here’s the first image I took, looking at the bird feeder as the sun was on its way down on the other side of the house:

ESP32-Cam with no IR filter
I tried to set the focus at where the bird feeder is. Not sure if I was entirely successful…

It’s not too bad. Things look a little weird – the trees, sky, and grass all look like they’re from a really cheap dream sequence. Two hours and fifteen minutes later, the sun had set but there was still plenty enough light for the camera:

ESP32-Cam

Twenty minutes later, the streetlights were all on, there was very little glow in the sky, and the image started to get pretty noisy:

ESP32-Cam

Another 25 minutes and it was dark:

ESP32-Cam
The bright dot is a light the next block over

So, with ambient light in the suburbs at night, the ESP32-Cam isn’t a stellar performer. This is not a high-end CCD rig.

I have a couple of inexpensive little IR illuminators but they’re so inexpensive that they didn’t come with specifications and I can’t find datasheets for them. I ended up using an LM317 to limit the current to about 125mA. I’m pretty sure they can go higher but figuring out how high is a project for another day.

IR Illuminator
Yep, one of the many, MANY illuminators out there that look just like this.

Here’s what that thing running at 125mA did:

ESP32-Cam
Well, I can see… something. Not much, though.
ESP32-Cam
After shining it around to find the best angle, I can juuuust make out where the bird feeder hook comes out of the ground.
ESP32-Cam
Yep, it’s definitely on…

Not so good. Or… was it not too bad? It’s not like I went out there with one of those 400 LED yard illuminators – this was a single diffused LED that’s pulling just over half a watt. Maybe I should revisit this with something that puts out a little more oomph.

The next thing on my list to check was how well it worked indoors. To test this, I put the camera and illuminator at one end of a hallway, closed the door, and turned off all the lights. My phone picked up a bit of light under the door at the far end of the hall, about 12 feet away:

Dark hallway with phone

And here’s what the ESP saw:

ESP32-Cam

And with the same illuminator (at the same power) turned on:

ESP32-Cam
You can even see the yellow sticky note I put on the door. Can’t see the smiley on it, though.
ESP32-Cam
My shirt is black under normal light.

Having the IR confined to an area where it can bounce around and not just be lost to the night makes a big difference. With a better (or more than one) source of IR, the ESP32-CAM may well make a decent indoor night camera. Tweaking some of the camera settings may improve things, too.

So… not the best performer, but for the cost (I think I paid nine dollars for this particular one, camera and all) and the sheer number of features that the ESP32 series has, I find it a pretty attractive little device.

I think the next ESP-ish things I want to look at are getting some pictures of the animals that wander through the yard at night, and maybe trying this particular experiment again with a bigger/better IR illuminator.

Recovering A Lost mdadm RAIDset In A Raspberry Pi-Based RAID Mirror

My previous post went over how to replace a failed USB-attached RAIDset member in a Raspberry Pi system running mdadm. Swapping out a failed RAIDset member is pretty simple, but what if the Pi or the micro SD card in the Pi fails?

Well, good news – mdadm stores metadata on the RAIDset members so even if you lose the RAID configuration on the SD card, you can recover it by scanning the disks.

I’m going to use the same Pi 4 (2GB) and two 64GB SanDisk USB sticks that I used in the previous post. They’re set up as a two disk mirror.

To simulate an SD card failure, I killed power to the Pi without shutting it down gracefully, then removed and re-imaged the card before putting it back into the Pi and powering it up. Here’s how to get your data back.

First things first – flash the appropriate Raspbian image to a micro SD card and set up your Pi like you would normally. Don’t forget to go through raspi-config and do the sudo apt update and sudo apt upgrade thing!

Once your Pi is up and running, fully up to date, and configured the way you like it, then shut it down, reattach the USB drives, and start it up again. Once it’s booted, check dmesg to see if the Pi can see the USB drives. The first USB drive will be called sda, the second will be sdb, and so on:

In my case, I only have two disks, /dev/sda and /dev/sdb and they’re both showing up. If you’re not seeing the disks, check connections and power and fix the problem before going on.

Using cat /proc/mdstat is the easiest way to check the status of your RAID array, but if you run it now, you’ll get nothing:

That’s because mdadm hasn’t been installed yet. Install it with the following:
sudo apt install mdadm

Once it’s installed, scan the disks on or attached to the Pi and create a RAIDset with any member disks found, using
sudo mdadm --assemble --scan --verbose

Notice that it found both /dev/sda and /dev/sdb, and it added them to an array.

Check the status of the array with cat /proc/mdstat:

It’s showing as [UU], which means both members are online and working properly. The array didn’t even need to resync!

Now, check the /etc/mdadm/mdadm.conf file to see if the configuration has been automatically updated:

Under “definitions of existing MD arrays” you can see the array is showing up. Also notice that it’s still called “DEV-04:0” – DEV-04 was the name of the Pi before I wiped out and rebuilt the SD card, so chances are it has the right data.

Reboot your Pi to see if it (and the array) come up cleanly. This isn’t a necessary step but I like doing it to make sure things are working and I haven’t forgotten anything.

Run blkid to see if /dev/md0 is listed:

So far, so good. Now, edit your /etc/fstab file, adding an entry for /dev/md0 but use the UUID instead of the device name:

I’m using /mnt as the mount point but it can be anywhere you want.

Once you’ve saved the file (and if everything is working properly), you should be able to mount the drive and use it with sudo mount -a

In this case, boring is good

If there are no errors and the system doesn’t hang or do something else weird, things may be working! Run df to see if /dev/md0 is mounted:

Hopefully you’ll see something similar. Check out the mounted directory and see if your data is there:

It’s there… it’s all there!

Depending on how you had things set up, you may need to change or set some permissions, but if it mounts and you can browse to it, that’s a good sign. Reboot the Pi once more to make sure the array and mounting work as expected, and if so, then everything should be good to go!

You can also use this method to move a RAIDset from one machine to another, although you should really, REALLY back up your data before you do that.

Actually, getting a backup of your stuff periodically isn’t a bad idea either. mdadm is mature and pretty stable, but a RAIDset with redundancy is no substitute for backups!

Replacing A Failed USB Disk In A Raspberry Pi-Based RAID Mirror

My previous post went into how to create a simple but functional NAS with a Raspberry Pi 4B and two USB-attached SATA disks. In the two weeks or so that it’s been running, the NAS I built has performed very well and has been reliable (hopefully I won’t regret typing that).

But what to do WHEN a disk fails? Disks fail – even that fancy new enterprise-grade SSD that cost an arm and a leg will fail at some point. The good news is that if you’re using mdadm to provide some kind of redundancy with your disks, things should still be working if a disk fails. The bad news is that unless you’ve got a RAIDset that can specifically tolerate more than one failure (like RAID 6), you need to replace that failed disk ASAP.

I’m confident that I’ll be able to recover from losing a disk in my shiny new NAS, but I’m not one to tempt fate so I built another RAIDset with a spare Pi and two 64GB SanDisk USB sticks to play around with instead. They’re slower than the disks so things like the speed the RAIDset syncs back up is going to be different than in my previous post.

So here’s the setup – it’s a Raspberry Pi 4B (2GB) with two 64GB USB flash drives in a RAID 0 (mirror) configuration.

Here it is, working properly, with the output of cat /proc/mdstat:

and checking to see if it’s mounted using df:

To simulate a disk failure, I removed one of the USB sticks while everything was running. Here’s the output of dmesg showing the disconnection and that mdadm is soldiering on with only one disk:

Looking at the list of USB-connected devices only shows one SanDisk device:

And now the output of cat /proc/mdstat is showing a failed disk (note the “U_”):

The good news is that yes, /dev/md0 is still mounted and usable, even though it’s in a degraded state.

I reformatted the USB stick on my Windows PC so the data that was on there was lost, then reconnected it to the Pi:

There are two SanDisk devices again.

And here’s the output of dmesg again – you can see the time difference between the failure and when the “new” disk was connected:

Note that the messages both of the failure and of the newly connected USB stick show them as sdb. It could just as easily have been sda, so make sure you check to see which one failed – and, more importantly, which one didn’t!

So now there are two disks connected again, but only one of them has the RAIDset data on it. In this case, sda is the one with the data that needs to be mirrored over. Again, it could’ve been sdb. For one last check, get the output of cat /proc/mdstat again:

Notice it says sda – that means that sda has the data we want to mirror over to the other disk, which, as the previous output of dmesg showed, is sdb.

If you are replacing a failed RAID member, the replacement must be the same size or larger than the failed member. That goes for any kind of RAID level and any type (i.e. disk mirroring or partition mirroring). Keep in mind that not all disks of the same stated capacity will actually have the same capacity, so make sure you do a bit of research before going out and spending your money on a new disk that won’t fit your current array!

Now that the disk is reconnected and showing up, copy the partition layout from the existing RAIDset disk to the new disk with the following command:

sudo sfdisk -d /dev/sdX | sudo sfdisk /dev/sdY

In this case, the existing disk is /dev/sda and the new disk is /dev/sdb:

This step isn’t needed if you’re mirroring disks (as opposed to mirroring partitions), but it’s a good idea to do it anyway – if there’s an error here, you certainly don’t want to go any further until you’ve fixed the problem.

If sfdisk worked and didn’t give you any errors, then you’re ready to add the new disk to the RAIDset with the following command:

sudo mdadm --manage /dev/md0 --add /dev/sdY

Where sdY is the new disk – in my case, sdb:

If you didn’t get any errors, run cat /proc/mdstat again and you’ll see your RAIDset is rebuilding:

Notice how it now shows that there are two active elements in md0sdb[2] and sda[0]? That’s a good sign. Keep checking every once in a while to make sure the recovery is progressing.

Once it’s done, the RAIDset should be showing as all “U” again:

If you see that, everything’s rebuilt and your RAIDset is ready to handle another disk failure.

Hopefully you never need to use this information, but if you do, I hope it helps!

Raspberry Pi 4-Based NAS Using USB-Connected Disks

Our little DNS-323 has been rock solid for the last decade but it’s getting long in the tooth and it’s just pokey enough to be annoying when I’m trying to do things on it. After a bit of consideration, I decided to see if I could build one with a Raspberry Pi.

It didn’t take a lot of research to find out that the Pi 1 through Pi 3+ aren’t particularly suited for NAS work. They may have the CPU horsepower, but with the on-board Ethernet and USB sharing the same USB2 port, their performance is reportedly not all that great.

The Pi 4, on the other hand, has entirely different hardware; the USB ports (of which there are two USB3 and two USB2) have their own controller, while the on-board Ethernet has its own controller. This makes for an ENORMOUS improvement in the performance of both USB:

Image from (2020-03-23): https://magpi.raspberrypi.org/articles/raspberry-pi-4-specs-benchmarks

and Ethernet:

Image from (2020-03-23): https://magpi.raspberrypi.org/articles/raspberry-pi-4-specs-benchmarks

With numbers like that, I thought it would be worthwhile to try building a little RAID1/mirrored home NAS around a Pi 4. Here’s what I used:

  • Raspberry Pi 4, 2GB model, qty 1
  • Official Raspberry Pi 4 power supply, qty 1
  • 32GB Sandisk UHS-1 MicroSD card, qty 1
  • 4TB Western Digital Blue 3.5″ hard drive, qty 2
  • Vantec NexStar TX 3.5″ external USB3 enclosure, qty 2
  • 5ft Category 5e patch cord, qty 1

I went with mirroring two disks (RAID1), so that is what I’m going to go through here. If you want to set up a single disk, or set up something like a four-disk mirror, or RAID5/6/1+0, you can use the same software I did but you’ll have to do a bit of research into the settings.

Oh, and if you are going to create a RAIDset with more than one disk, make sure they’re all the same size, otherwise the mirror will only be as large as the smallest of the disks that are part of the RAIDset!

I had originally planned to use openmediavault to mirror the disks and create the network shares, but unfortunately it doesn’t support USB-connected disks. That’s what I get for not reading enough before I buy stuff, I suppose. With omv out of the picture, I decided to try mirroring the disks and set up the network shares myself. Here’s how I did it:

Part 1: Set Up Your Raspberry Pi

I’m not going to get into this because there are already a ton of sites out there that will show you how to do this (and describe it better than I can). I built my NAS with the Raspbian Buster Lite image, dated 2020-02-13. Do not use wireless (don’t bother with a wpa-supplicant.conf file), but make sure you enable ssh, go through the raspi-config menu and don’t forget to apt update and upgrade!

Oh, and CHANGE YOUR PASSWORD!

Part 2: Assemble And Format The Disks

This part’s pretty easy. Install the hard drives in the enclosures, connect them to your PC, and using the software of your choice, remove any existing partitions on the disks and create a single NTFS partition (or ext4 if your PC is a Linux machine) that uses the entire capacity of the disk. Once that’s done and you get no errors, safely remove the enclosures from your PC.

Part 3: Put Together And Connect Your Hardware

This part’s pretty easy too. Connect the USB disks to the Pi and turn them on. If things are working properly, the physical enclosures and the disks will be present. To check if the Pi sees the enclosures, type lsusb:

Two USB to SATA devices, check.

To see if the actual spinning hard drives have been detected, type dmesg | grep sda for the first enumerated disk, and dmesg | grep sdb for the second:

The second disk (/dev/sdb) should look pretty much the same.

Do not go any farther if the output of either of those commands doesn’t look correct, or if the disk capacity listed is different than you expect. Go back, check all of the parts and connections, and try again.

Part 4: Set A Static IP On eth0

You should at this point already have a cable connecting your Pi to your router or a switch. If you’re using wireless… well, I suppose you can do that if you really want or need to, but you’re going to be making your Pi do its work with one foot in a bucket. Use that RJ-45 jack and get yourself some nice clean Gigabit Ethernet goodness.

To set a static IP, use your favourite editor to edit the /etc/dhcpcd.conf file. Make sure to use sudo so you’re editing it as the superuser. Go right to the bottom of the file, and add the following lines:

# NAS Static IP for eth0
interface eth0
static ip_address=X.X.X.X/YY
static routers=Z.Z.Z.Z
static domain_name_servers=A.A.A.A B.B.B.B

Where:

  • X.X.X.X is the static IP address on your network that you want your NAS to be reachable at
  • YY is the CIDR representation of your subnet mask (most home or small businesses will be /24)
  • Z.Z.Z.Z is the IP address for your gateway/router
  • A.A.A.A is the IP address for your primary DNS server
  • B.B.B.B is the IP address for your secondary DNS server (if you have one)

Make sure that you type everything exactly. Even if you only have one router and one DNS server, you still need to type static routers and static domain_name_servers with the “s”.

Once you’ve finished setting the dhcpcd.conf file, reboot your Pi. Once it comes back up, see if you can ping other devices both inside and outside your network. Then try to do another apt update and upgrade to see if your Pi can talk to the Raspbian repositories. If not, go back over the file and make sure the changes you made were saved. Also check to make sure everything is spelled correctly (remember it’s case sensitive).

Part 5: Mirror The Disks

Congratulations! If you’re here, that means you have successfully set up your Raspberry Pi, can see it on the network, and have two hard disks connected via USB. You are now ready to do something not a lot of other people do – use a Raspberry Pi to make a RAIDset out of a pair of USB-connected disks. Fortunately, the software (like the hardware) has made leaps and bounds since the last time I tried it and it’s pretty easy to set up.

First, let’s make sure the right stuff is installed:

sudo apt update
sudo apt upgrade
sudo apt install mdadm

Since you already made sure the disks were working in Step 2, you can go ahead and create a RAID1 mirror. In my case, I didn’t care about the partition size so I used the entirety of both disks with the following command:

sudo mdadm --create --verbose /dev/md0 --level=mirror --raid-devices=2 /dev/sda /dev/sdb

In this case, mdadm is being told:

-- create : Make a new RAIDset
-- verbose : Show what’s going on while the command is running
/dev/md0 : The name of the RAID device you’re creating
--level=mirror : Create a mirror (RAID1)
-- raid-devices=2 : How many disks will be used
/dev/sda /dev/sdb : The names of the disks that will be used

Again, this is how I set up my own little two-disk mirror. If you have a different number of disks or want to set up a different kind of RAIDset, the syntax is pretty much the same but the options are different. You may also want to use particular partitions instead of entire disks like I did. Check out the mdadm man page.

You should now see the lights on the enclosures blinking furiously and/or be able to feel/hear the the hard drives doing something. Depending on the kind and size of disks you have and the type of enclosure, creating the mirror and syncing it up may take up to a day. You can check the progress with the following command:

cat /proc/mdstat

which should give you output something like this, which shows you the status of the mirror sync:

Personalities : [raid1]
md0 : active raid1 sda[3] sdb[2]
3096885440 blocks super 1.2 512K chunks 1 near-copies [2/2] [UU]
[>....................] resync = 0.2% (61504/3096885440) finish=398min speed=14298K/sec
unused devices: <none>

Once it’s done, instead of the above, you should see something similar to the following when you run the same command:

pi@PI-0:~ $ cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda[1] sdb[2]
3906885440 blocks super 1.2 [2/2] [UU]
bitmap: 0/30 pages [0KB], 65536KB chunk

Note the line with the [UU] at the end. Each U represents an active and healthy RAID disk. If you run cat /proc/mdstat and you see a _ (underscore) instead of a U, there’s a problem with a disk that requires your immediate attention.

Now that the RAIDset is built, you need to save its configuration so your Pi knows what to do with it when it boots:

sudo -i
mdadm --detail --scan >> /etc/mdadm/mdadm.conf

exit

Now confirm it was saved:

cat /etc/mdadm/mdadm.conf

and look for the line that says something like this (obviously your UUID will be different):

ARRAY /dev/md/0 metadata=1.2 UUID=061a78a9:ceadf64b:b124c1d4:7e35ae85 name=PI-0:0

Now reboot your Pi, and once it comes back up, use cat/proc/mdstat and blkid to see if everything’s okay:

Seeing [UU] and /dev/md0 is a good sign

Part 6: Create A Filesystem

Now that the disks are mirrored, it’s time to put a filesystem on them. I use ext4, and I’m creating a filesystem on the RAIDset, NOT the physical disks, so the command is:

sudo mke2fs -t ext4 /dev/md0

This may also take a little while. Once it’s done, edit the /etc/fstab file so that the filesystem on the RAIDset will automount at boot. I use the /mnt directory as the mount point, here’s what my fstab file looks like:

Notice that the UUID of the partition is the same as the UUID for /dev/md0 in the output of blkid.

Type the following to see if your fstab file is set up right:

sudo mount -a
df

You should see something like this:

In this case, /dev/md0 is mounted at /mnt and everything looks good. The use is at 12% because I’ve already been using the NAS – yours will probably say 0% or 1%.

Now, shut your Pi down, then turn off the disks. Turn on the disks, wait for them to spin up, and boot up the Pi. Run those two commands again and make sure everything looks good before going any further.

Part 7: Set Up A File Share With Samba

Samba is a mature, stable, and very useful batch of software that makes it pretty easy to create simple network shares. It may already be installed, but just to be on the safe side:

sudo apt install samba
sudo apt install samba-common-bin

Once it’s installed, you’re going to need to configure it by editing the /etc/samba/smb.conf file. I’m only going to set up one network share for now, and if you’re new to Samba, I suggest sticking to one share until you’re familiar with it.

Before we edit the file, though, we need to create the directory that Samba will use to share over the network:

sudo mkdir /mnt/NAS_FILE
sudo chown pi /mnt/NAS_FILE
sudo chgrp users /mnt/NAS_FILE

Do NOT use /mnt as the directory for your file share – always use a directory that resides on the device you’re mounting. If for some reason /dev/md0 doesn’t mount properly, you may end up writing data to and filling up the SD card instead of using the disks!

The default smb.conf file contains a number of examples, including one for a network share and one for a print share. Copy the default file:

sudo cp /etc/samba/smb.conf /etc/samba/smb.conf_OLD

Now, with sudo, use your favourite text editor to open /etc/samba/smb.conf and go to the bottom section of the file, labelled “Share Definitions”. Delete everything in that section and replace it with the following:

[NAS]
comment = NAS Fileshare
path = /mnt/NAS_FILE
browseable = yes
read only = no
writable = yes
create mask = 0775
directory mask = 0775
valid users = pi

So the “Share definitions” section in my smb.conf file looks like this:

Notice how the “valid users” section has the name “pi” in it – you can change that to anyone you’d like (or have more than one user on that line), but for each user on that line, you’ll need to create an account on the Pi for them. I just stuck to the pi user because I wanted to keep things simple to start.

To test your smb.conf file, run the following:

testparm

The “Loaded services file OK” is a good sign that your smb.conf file has no obvious errors in it. If you don’t get that message, go back through the file and make sure everything is spelled properly, etc.

Now, for each user account you want to grant access, you need to run the smbpasswd utility to set them up in Samba. It will ask for a password, and that password really, really should really be different than the password that’s used by the user to log into the Pi itself!

To add someone to the Samba system:

sudo smbpasswd -a USERNAME

To disable someone’s Samba account:

sudo smbpasswd -d USERNAME

To re-enable someone’s Samba account:

sudo smbpasswd -e USERNAME

In my case, I ran:

sudo smbpasswd -a pi

and gave it a password that was different from the password that I use for logging in with the pi user.

At this point, Samba is installed, you’ve created a directory on the RAIDset that Samba will use for the file share, you’ve edited the smb.conf file, ran smbpasswd for every user that’s listed in the smb.conf file, and tested your configuration with testparm. It’s time to restart Samba so it loads the new configuration:

sudo service smbd restart

If you don’t get any messages or errors, things may actually be working!

Part 8: Access Your New NAS

Finally, the payoff – your own home-built NAS! How you will access it depends on the operating system on the computer you want to access it with. If you open your file or network browser, it may automatically show up. Otherwise, you will have to browse to it. Open your file or network browser and browse to the static IP address you set way back in Part 4.

In Windows, it should look something like this:

In Ubuntu, you may have to enter smb:// before the address:

When you try to open the NAS share, you should be prompted for a username and password. The username will be:

localhost\USERNAME, which in my case was
localhost\pi

…and the password will be whatever you set it to with smbpasswd.

Some of my friends and family may disagree with this statement, but I like it when things are organized. To help with this, I designed and printed a lightweight frame that holds two disks and a Pi, and has several holes for 5mm M3 screws to fasten things like cable management or velcro or whatever to it. Here’s how mine turned out:

NAS Frame

I put the model on Thingiverse, it’s at https://www.thingiverse.com/thing:4235287

As for performance… this setup is much more responsive and can transfer files to and from the disks much faster than the old NAS. Disk fragmentation can slow things down, and the old NAS is a decade old, but it was only about half full so that shouldn’t have been too big a problem.

Transfer rates to and from the Pi are faster than my home wireless is, and browsing the directories on the file share is no different than browsing the directories on my PC. Nice and quick.

150Mbps isn’t too shabby at all! The best I could get out of the old NAS was around 70Mbps. Bottom line – a Pi 4 with two external USB3-connected hard drives makes a serviceable and reasonably fast NAS for home or small business use, although there are security considerations that need to be addressed prior to using it out in the real world.

So that’s the deal. I did a bunch of torture testing when I first set things up, and things recovered gracefully. I will do another post soon to discuss how to fix a mirror if there’s a disk failure or if you need to recover the array entirely due to a Pi failure.

I hope you found this useful!

When Is A Clog Not A Clog?

My printer has been a little fussy lately. I was printing some PETG and didn’t notice any problems, but as soon as I switched to PLA the flow of the plastic out the nozzle would slow down for a bit, then go back to normal, then slow down again.

I’ve been running a 0.6mm nozzle for a while now and while it’s bigger than the stock nozzle, it can still clog, so I went through all of the usual steps – pulls at various temperatures, using the cleaning needle, trying to force the filament through by hand, but nothing seemed to fix the problem.

It was acting different than the clogs I was accustomed to, too. Normally a clog is kind of consistent – filament comes out in a spiral or diagonally or some other strange pattern. This time it was coming out nice and straight, and I didn’t notice any problems when running the extruder 10mm at a time.

Once I tried to extrude more than about 20mm, things started to act weird. Filament would come out the nozzle properly, then stop for a second or two while the extruder motor teeth turned and ground against the filament, then flow normally again. I was out of clean nozzles to try and figured that I was going to have to order another one, but the pausing had me thinking.

There are some differences between PLA and PETG. PETG usually runs at a higher temperature than PLA, but I’d even tried doing pulls with PETG to clean the nozzle and had no luck. But… I usually print PETG at about 30mm/s, while I print PLA around 50mm/s.

I thought about it some more and the next thing I came up with was that the hotend on the printer wasn’t heating enough, or was heating and cooling in cycles. It’s the same one that came with the printer, but after confirming the temperature on the display with a non-contact thermometer, I was pretty sure that wasn’t the problem.

Then I thought about the filament path from the extruder motor to the nozzle tip. On the CR-10s (and a lot of other printers), a PTFE Bowden tube connects the extruder motor assembly to the hotend. I wondered if the hotend and nozzle weren’t the problem at all. I slowly extruded filament until it was flowing out of the nozzle, then marked on the filament where it went into the extruder, then pulled it out. Then I reinserted it by hand and sure enough, there was resistance before the line on the filament had made it back to where it had been. So something was jamming it before it got to the nozzle.

I did some more pondering and then figured it out – the filament was jamming just as it was entering the hotend. As the filament heated up and melted, it went past the jam very easily, until unmelted plastic got to the same spot and jammed. A second or two later there was enough heat to melt it again, so it moved again. The reason PETG printed and PLA didn’t was because the hotend could melt the filament back far enough at the low print speed of the PETG to keep it moving, but with PLA there was too much cold filament too quickly.

A bit of swearing later (I no longer have a fingerprint on my left index finger), I had removed the Bowden tube from the hotend. This is what I found:

Melted Bowden Tube
That doesn’t look right…
Melted Bowden Tube
Ah-HA! Melted inwards. A little darkened, too.

The Bowden tube had melted inward, partially blocking the tube. I don’t know how long this has been going on, but a quick check of my notes showed that since I got the printer I’ve been running it more and more slowly, so it may have been a while.

A quick (and carefully measured) straight cut to freshen up the end and it went back into the printer… and it’s been printing beautifully since then. I haven’t tried upping the speed back to 70-80mm/s with PLA but I may do some experimenting with that soon.

Not bad for a year-old piece of plastic tubing that’s constantly exposed to temperatures above 230C. The printer came with a spare just in case, but I did some looking around and there are aftermarket tubes that are rated for higher temperatures. Might be something to look into at some point.

Long story short: if your printer is printing in a weird stop/go pattern and the extruder motor is running properly, it may be the tube, not the nozzle.

Oh, and this is as good a point as any for a reminder – if you have a 3D printer, INSTALL A SMOKE DETECTOR NEARBY. Melting plastic is generally good when printing, as long as it’s the plastic you intended to melt!

Temperature And Humidity Display Using Arduino, DHT22, And MAX7219 Display

I finished another project today. This time it’s a simple temperature and humidity display, and so far it’s working pretty well. It’s built around an Arduino Nano and uses a DHT22 sensor. The display is an extremely cheap MAX7219-based four-module LED matrix (130x32mm), and its brightness is controlled by a capacitive touch sensor (11x15mm).

In this case, everything runs off a 5V supply so there are no level shifters needed and a single USB cable can power the whole thing. Any other 5V Arduino with hardware SPI will work fine here, too. The software libraries I used also support software SPI but I haven’t tried that out.

Here’s the layout:

  • Everything is connected to the +5V and GND pins on the USB connector.
  • MAX7219 CLK to Arduino 13.
  • MAX7219 DATA to Arduino 11.
  • MAX7219 CS to Arduino 10.
  • DHT22 I/O to Arduino 2.
  • Touch sensor I/O to Arduino 4.

I had everything on a breadboard but forgot to take a picture before wiring everything up to fit in the case… here it is wired up and just before being prepared to put into the case.

Arduino DHT22 ST7735

The program uses the MD_MAX72XX and MD_PAROLA libraries for the display, and the SimpleDHT library for the DHT22. It took me a while to wrap my head around the MD_PAROLA stuff, but the examples included with the libraries were very helpful. Here’s the program:

/* Temp and RH DHT22 MAX7219 for Dot 04
 *  Uses Nano to check DHT22 and display on 8x8 dot matrix (x4) MAX7219.
 *  Meant to be used indoors.
 *  Has two brightness settings, 4 and 15 (on scale of 0-15)
 *  Runs off 5V USB.
 *  MAX7219 controlled by MD_Parola and MD_MAX72xx libraries
 *  DHT22 using SimpleDHT
 *  PINS:
 *  DHT22 data: D2
 *  MAX7219 clock: D13, data: D11, CS: D10
 *  Intensity: D4
 *  Uses a MAX7219 32x8 LED module from Banggood. Hardware type is MD_MAX72XX::ICSTATION_HW, 4 devices
 *  Puts temp and RH on display at same time
 */

/*
 * MAKE SURE YOU RUN THE MD_MAX72XX_HW_Mapper to confirm the hardware setting for your particular display!
 * The results I got for the display I have were:
 * 
 * HW_DIG_ROWS 1
 * HW_REV_COLS 1
 * HW_REV_ROWS 1
 * Your hardware matches the setting for IC Station modules. Please set ICSTATION_HW.

 * 
 */

#include <MD_Parola.h>
#include <MD_MAX72xx.h>
#include <SimpleDHT.h>
#include <SPI.h>

// Set up DHT22 vars for data TX/RX
#define h_w 8
#define h_h 8
static unsigned char h_w_bits[] = {
   0x3c, 0x42, 0xa5, 0x81, 0xa5, 0x99, 0x42, 0x3c };

#define s_w 8
#define s_h 8

static unsigned char s_w_bits[] = {
   0x3c, 0x42, 0xa5, 0x81, 0x99, 0xa5, 0x42, 0x3c };

// Create instance for the DHT22 using pin 2 for data xfer
SimpleDHT22 dht22(2);


// Define the number of devices we have in the chain and the hardware interface
// NOTE: These pin numbers will probably not work with your hardware and may
// need to be adapted
#define HARDWARE_TYPE MD_MAX72XX::ICSTATION_HW  // Found using the MD HW mapping program

#define MAX_DEVICES 4 // Four 8x8 modules on this particular board

#define CLK_PIN   13
#define DATA_PIN  11
#define CS_PIN    10

#define BRIGHT_PIN 4

// Hardware SPI connection
MD_Parola P = MD_Parola(HARDWARE_TYPE, CS_PIN, MAX_DEVICES);

byte CountUp = 0;

void setup() {

  delay(500); // Need this because display doesn't seem to start up right away.

  P.begin(2); // Using 2 zones, one for temp, one for humidity

  pinMode(BRIGHT_PIN, INPUT);
  
  P.setZone(0,0,1);
  P.setZone(1,2,3);
  
  P.displayZoneText(0, "Hi!", PA_CENTER, 75, 0, PA_PRINT, PA_NO_EFFECT);
  P.displayZoneText(1, "Hi!", PA_CENTER, 75, 0, PA_PRINT, PA_NO_EFFECT);

  P.setZoneEffect(0, 1, PA_FLIP_UD);  // Need this because I glued the display in upside down >:-(
  P.setZoneEffect(1, 1, PA_FLIP_UD);  // Need this because I glued the display in upside down >:-(
  P.setZoneEffect(0, 1, PA_FLIP_LR);  // Need this because I glued the display in upside down >:-(
  P.setZoneEffect(1, 1, PA_FLIP_LR);  // Need this because I glued the display in upside down >:-(
  P.displayAnimate();    
  delay(2000);
}

void loop() {

  jumpback:

  // If touch sensor is active, cycle through the 16 levels of brightness until sensor is inactive.
  int brightness_change = digitalRead(4);
  while (brightness_change == 1){
    if (CountUp == 16){
      CountUp = 0;
    }
    P.setIntensity(0, CountUp);
    P.setIntensity(1, CountUp);
    delay(250);
    CountUp = CountUp + 1;
    brightness_change = digitalRead(4);
  }

  float temperature = 0;
  float humidity = 0;
  int err = SimpleDHTErrSuccess;
  if ((err = dht22.read2(&temperature, &humidity, NULL)) != SimpleDHTErrSuccess) {
    // If we're here, there was a problem reading the DHT22. Show an error then try again.
    P.displayZoneText(0, "Dht", PA_CENTER, 75, 0, PA_PRINT, PA_NO_EFFECT);
    P.displayZoneText(1, "Err", PA_CENTER, 75, 0, PA_PRINT, PA_NO_EFFECT);
    P.displayAnimate();
    delay(5000);
    goto jumpback;  // I know, I know. Don't say it...
  }

  // Convert the float to a string to display
  char temp_result[6];
  dtostrf(temperature,2,0,temp_result);

  // Convert the float to a string to display
  char hum_result[6];
  dtostrf(humidity,2,0,hum_result);

  P.displayZoneText(1, hum_result, PA_CENTER, 75, 0, PA_PRINT, PA_NO_EFFECT);
  P.displayZoneText(0, temp_result, PA_CENTER, 75, 0, PA_PRINT, PA_NO_EFFECT);
  P.displayAnimate();

  delay(3500);  // DHT22 max sample rate is about 2 seconds.
}

If you are using a MAX7219-based display, save yourself some time and frustration by connecting it and running the MD_MAX72XX_HW_Mapper program that comes with the MD_MAX72XX library before you do anything else. It will tell you how your display is set up, regardless of how it actually looks.

After doing some testing, I found that the capacitive touch sensor I was using could reliably detect my finger out to about 5mm away. That was great because then I could hide it inside the case and there’d be no switch, no pad… just a “magic” spot on the back that changes the brightness if you put your finger there.

I designed a case for this particular project, including the specific display and touch sensor I had on hand. It’s vented, has a hole for a USB cable, and is closed up with four 6mm M3 screws:

Arduino DHT22 MAX7219
Fresh off the printer… with a bit of over-extrusion.
Arduino DHT22 MAX7219
The capacitive touch sensor in its dedicated spot right in the middle of the back panel.

With everything wired up and tested, I hot-glued everything… and I mean everything. Every connector, every module (except the Nano’s mini-USB port – never know if I’ll want or need to reprogram it) … it’s all quite secure inside the case. I glued put a piece of plastic on the back of the display just in case any other parts work their way loose and came in contact with it. I’m still a little wary of doing things this way, but it sure beats drawing up and etching boards for this kind of stuff!

Once the glue had cooled and I confirmed everything was stuck good and tight, I closed up the case and plugged the cable into a 5V USB power supply. The LEDs flashed, and then… everything was upside down. I’d glued the display in upside down.

So… another 45 minutes or so of pondering and looking and I found how to flip the display in software so it looked right again. If you run into this problem, check out setZoneEffect() in the MD_PAROLA documentation.

Here it is, from the back:

Arduino DHT22 MAX7219

And from the front, display pointing the right way:

Arduino DHT22 MAX7219

The STL files for the case are available at https://www.thingiverse.com/thing:4202464

Find A File That Contains A Particular Text String (Windows)

Windows File Explorer is many things. One thing it is not, however, is good at searching. It used to be… but somewhere along the way something got broken.

Fortunately, there’s a command-line tool that works much better than the GUI, called findstr. You can look up the options by typing

findstr /?

If you’re looking for a particular text string and you only want to know which files contain that string, use the following command:

findstr /S /I /M /C:TEXT_STRING_HERE *

The /S means search this directory and all subdirectories.
The /I means don’t be case-sensitive.
The /M means print only the filename if there’s a match.
The /C: specifies the text string you’re looking for.
The * means check everything.

Fix For Arduino IDE Compiling Very Slowly In Windows

My Raspberry Pi has been compiling Arduino programs in about a tenth the time that my main Windows machine has. The Windows machine is not a slouch – i7-7700HQ with 16GB RAM and a SSD, and it was getting annoying waiting for several minutes while some of the larger ESP programs compiled.

Turns out the antivirus was running full-tilt, scanning every file as it was loaded into the compiler, pinning the CPU.

The fix for me was to set an exception in my AV software for the C:\Users\[yourname]\AppData\Local\Arduino15 folder. It made a huge difference in speed and my laptop no longer sounds like a jet engine when it’s compiling a 20-line program for a microcontroller with 2.5kB of RAM…