Table of Contents
Harder Soft-Unbricking a Ubiquiti Unifi UAP-Pro AP
Re-deploying the Ubiquiti Unifi UAP-Pro with OpenWRT (instead of Unifi) at the new base-camp revealed a problem: it simply didn't work anymore. No DHCP reaction, no default IP, nothing. Except for the LED blinking white after a reset, it seemed completely broken. Opening the case and attaching a terminal to the serial console, revealed bootloader output that wasn't promising but at least it wasn't completely dead. So we had to get our hands dirty, pop the hood and hack around to fix this jffs2 corruption issue and revive the AP without costly and probably long/painful RMA. Maybe this knowledge can help you too.
The Symptom
U-Boot unifi-v1.2.3.74-ge597862a (Mar 14 2013 - 18:30:11) Board: Ubiquiti Networks AR9344 board (e507-27.2122.0030.0030) DRAM: 128 MB Flash: 16 MB Net: eth0 Hit any key to stop autoboot: 0 Scanning JFFS2 FS: .| Unknown node type: e008 len 52 offset 0xf41eec [...] done. cant find .firewall.uci-mWgTyx�� [...] cant find .wireless.uci-HxpWh9 cant find .network.uci-anmaB1 ����������������������������������������������������������������������� ����������������������������������������������������������������������� ����������������������������������������������������������������������� �������������������������������������������������������
It is still unclear what exactly causes this jffs2 corruption, but there have been at least three other reported cases and one directly confirmed by djs500, who found the draft-wip-template for this mission log on the net, had exactly the same problem and came back with positive feedback that he was able to fix it using Solution 3, as we did too.
Solution 1
Set a static IP on your PC's NIC from 192.168.1.0/24 range, but not 192.168.1.20 (this is the Unifi AP default TFTP IP).
Follow the steps to unbrick your UAP:
- Set the IP on your PC to be able to access the bricked unit and prepare the firmware file.
- On your PC, open your TFTP client and locate the firmware.bin file so you can start it later. Do not initiate the transfer.
Using the Windows integrated TFTP client the command to prepare is: “tftp -i 192.168.1.20 PUT [path to file]\firmware.bin” Unplug the bricked unit. Plug the LAN connection of the PoE injector directly to your PC's NIC. Keep the UniFi AP's reset button depressed and plug in network/PoE in the unit. Keep the reset button depressed until you see the light cycling relatively fast through amber/green/off colors (~14 seconds from power on) → Release it. Now the device is in TFTP transfer mode. Engage the TFTP push command and wait. The device will write the firmware and it will reboot. If you wait too long to start the TFTP transfer, the push will not work as the device will stall. Please redo from step 3. To ensure all goes clean, after the device gets stable (blinking or steady amber LED), give it a reset: Remove power. Reconnect holding the reset button for ~7 seconds (green led will flash once) → release reset button and wait for the device to stabilize again. After device is stabilized, power cycle it again, and you are good to go on adopting the device in the controller again.
Official UBNT solution didn't work at all
This may be due to the fact that this workflow expects a bricked UBNT firmware and NOT an OpenWRT installation.
Solution 2
Soft Unbrick via Serial using urescue and a tftp server was another method mentioned somewhere, but sadly it never really flashed anything and went staight to the reset in less than a second instead. It could never have written the complete firmware in this short time period and after testing: it didn't, so “Firmware update complete” was a lie:
ar7240> urescue Setting default IP 192.168.1.20 Starting TFTP server... Using eth0 (192.168.1.20), address: 0x81000000 Waiting for connection: - Receiving file from 192.168.1.254:39067 Received 4683453 bytes Firmware Version: BZ.ar934x.v3.2.1.2601.140606.1622 Setting U-Boot environment variables Will not overwrite u-boot partition! Skipped. Firmware update complete. Resetting..
Serial urescue didn't work at all
Solution 3
None of the above solutions worked with our OpenWRT UAP-Pro, so we had to grab one of the original jffs2 images from a live one with original UBNT firmware, use a tftp server to transfer that jffs2 image file to the AP, unlock and erase the corrupt jffs2 image on the flash chip and eventually write the new image by hand.
The following unbrick trace documents these 9 easy to follow steps codec, fpletz and I took to restore the UAP to its orginal UBNT firmware with default configuration (and default ubnt login) to be able to finally reinstall OpenWRT.
Step 1: Preparation
If you've lost your original firmware backup too, you can download the jffs2 image here:
jffs2.img.bz2
Just bunzip2 the downloaded jffs2.img.bz2 and put jffs2.img into your tftproot so that the AP can access it via LAN. If you don't know which tftp-server to pick, dnsmasq and atftp have been reliable alternatives and have a solid base of documentation and HOWTOs to get started out there.
In this setup dnsmasq with tftp enabled was hosted on a laptop, eth0 IP 192.168.1.254 connected via LAN to the main port of the UAP-Pro.
Step 2: Connect Serial Console
Step 3: Transfer jffs2 image to AP
Power up the AP and press a key to enter u-boot menu, then transfer and load the original jffs2 image over tftp to the AP and store it in RAM, starting at memory address 0x83000000:
ar7240> tftp 83000000 jffs2.img
If the tftp server is reachable, the output should look like this:
Using eth0 device
TFTP from server 192.168.1.254; our IP address is 192.168.1.20
Filename 'jffs2.img'.
Load address: 0x83000000
Loading: #################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#############################
done
Bytes transferred = 16121856 (f60000 hex)
The firmware image is stored in RAM now and occupies 0xf60000 bytes. Writing it from there to flash memory directly may fail when the target area has not been erased or is write-protected. Let's take care of that first.
Step 4: Reset partition table to defaults
This gives you an overview of the partition table layout
ar7240> mtdparts
device nor0 <ath-nor0>, # parts = 5 #: name size offset mask_flags 0: u-boot 0x00040000 0x00000000 0 1: u-boot-env 0x00010000 0x00040000 0 2: jffs2 0x00f60000 0x00050000 0 3: cfg 0x00040000 0x00fb0000 0 4: EEPROM 0x00010000 0x00ff0000 0 active partition: nor0,0 - (u-boot) 0x00040000 @ 0x00000000 defaults: mtdids : nor0=ath-nor0 mtdparts: mtdparts=ath-nor0:256k(u-boot),64k(u-boot-env),15744k(jffs2),256k(cfg),64k(EEPROM)
Reset partition table to defaults with
ar7240> mtdparts default
Save environment changes:
ar7240> saveenv
Saving Environment to Flash... Un-Protected 1 sectors Erasing Flash.... done Erased 1 sectors Writing to Flash... write addr: 9f040000 done Protected 1 sectors
Step 5: Unlock Flash-Banks
Now it is time to make sure that any write-protection is disabled:
ar7240> protect off all
Un-Protect Flash Bank # 1
Step 6: Erase the corrupt jffs2
Figuring out the correct address and length was a bit difficult, unclear, uncertain, not well documented and took us quite a while, because we didn't want to brick it more by writing stuff somewhere other than we actually intended. Unfortunately, there was no documentation during the process of how we derived at that conclusion, but in the end we finally decided to start at address 0x9f050000 for the length of the jffs2 image as indicated by the tftp transfer (0xf60000) and the mtd partition size (0xf60000).
ar7240> erase 0x9f050000 +0xf60000
...................................................................................................................................................................................................................................................... done Erased 246 sectors
Step 7: Flash original jffs2
In this step we're going to copy the jffs2 image stored in RAM, starting at address 0x83000000, and write it to the previously erased jffs2 flash area starting at address 0x9f050000 for the length of 0xf60000. The cp command knows about flash memory areas and will automatically invoke the necessary flash programming algorithm, when the target area is in flash memory.
ar7240> cp.b 0x83000000 0x9f050000 0xf60000
Copy to Flash... write addr: 9f050000 done
Step 8: Verify the jffs2 was flashed correctly
Since we still have the jffs2 image stored in RAM, starting at address 0x83000000, we can now comfortably byte-compare it with the content starting at address 0x9f050000 (Flash) for the length of 0xf60000. Both have to match, otherwise something obviously went wrong during the flashing process.
ar7240> cmp.b 0x83000000 0x9f050000 0xf60000
Total of 16121856 bytes were the same
Step 9: Reset, Reinstall OpenWRT and enjoy your revival work
ar7240> reset
Resetting... U-Boot unifi-v1.2.3.74-ge597862a (Mar 14 2013 - 18:30:11) Board: Ubiquiti Networks AR9344 board (e507-27.2122.0030.0030) DRAM: 128 MB Flash: 16 MB Net: eth0 Hit any key to stop autoboot: 0 Scanning JFFS2 FS: . done. ## Booting image at 81000000 ... Image Name: MIPS Ubiquiti Linux-2.6.32.33 Created: 2014-06-06 23:26:31 UTC Image Type: MIPS Linux Kernel Image (lzma compressed) Data Size: 4450109 Bytes = 4.2 MB Load Address: 80002000 Entry Point: 80002000 Verifying Checksum at 0x81000040 ...OK Uncompressing Kernel Image ... OK Starting kernel ...