This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
playground:mission-log-template1 [2014/11/04 10:13] – [Local Spark-Cloud Server Dataflow] chrono | playground:mission-log-template1 [2015/07/07 12:20] – chrono | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== HOWTO: Set up and use a local Spark-Core Cloud ====== | ||
- | It took quite a bit of tinkering and a couple of clarifying sessions on IRC (Thanks to lbt and aholler for their input and support), to deploy the local Spark-Cloud test setup and interpret/ | + | {{tag>ethernet switch review hardware test energy}} |
- | + | ||
- | Well, the IoT (Internet of Things) is a hip buzzword these days and the [[mission: | + | |
- | + | ||
- | For all we know, it is at least save to assume, that we actually have no way of knowing how far this technology branch is going to develop and spread in a couple of years, just like the Internet itself 20 years ago. We should look at the privacy aspects before it's actually too late to do so. In the end, it boils down to this question: | + | |
- | + | ||
- | //Do we really want to give out our complete sensory data (sys/ | + | |
- | + | ||
- | In the year 2014, in a post [[http:// | + | |
- | + | ||
- | The current software implementation (firmware- and server-side) has no concept of mesh/p2p or direct networking/ | + | |
- | + | ||
- | ==== Remote Spark-Cloud (AWS) Dataflow ==== | + | |
- | {{: | + | |
- | + | ||
- | In this picture the blue lines represent the data flow of the Cores, the clients and the central server. All points marked with a red C show where the current implementation/ | + | |
- | ==== Local Spark-Cloud Server Dataflow ==== | + | |
- | {{: | + | |
- | + | ||
- | When you follow this howto and secure your network access with a strong VPN you'll end up with something that looks like this image, where we effectively mitigate all these issues and take back control of our privacy & autonomy. | + | |
- | + | ||
- | ==== Key Features/ | + | |
- | + | ||
- | |:::^ Remote Spark-Cloud (AWS) ^ Local Spark-Cloud Server^ | + | |
- | ^ Easy start with WebIDE | Yes | Manual setup required | | + | |
- | ^ Offline Development with Atom and git | No (Use WebIDE) | Yes | | + | |
- | ^ Number of exposable functions | 4 | unlimited((Since source code is open, this can be changed to whatever we like.)) | | + | |
- | ^ Number of exposable variables | 10 | unlimited((Since source code is open, this can be changed to whatever we like.)) | | + | |
- | ^ Local Cross-Compiler Toolchain needed | No | Yes | | + | |
- | ^ Cores still work when Internet is offline | No((Remember: | + | |
- | ^ Perfect Privacy/ | + | |
- | ^ Core Firmware can be trusted | Potentially Not | **Yes** | | + | |
- | ^ OTA Update capability | Yes (But potentially insecure) | **Yes** | | + | |
- | ^ Minimum Avg. Non-US Network Latency | >100ms | <10ms (LAN/WiFI) | | + | |
- | + | ||
- | ===== ===== | + | |
- | + | ||
- | ===== Installation ===== | + | |
- | + | ||
- | ==== Dependencies ==== | + | |
- | + | ||
- | === dfu-util === | + | |
- | + | ||
- | To claim, configure and flash our Cores locally we'll need dfu-util >= 0.7.0 | + | |
- | + | ||
- | < | + | |
- | $ emerge -av dfu-util | + | |
- | </ | + | |
- | + | ||
- | In this guide I've used the 9999 gentoo ebuild, which pulls and builds | + | |
- | the current git master, on other distros your mileage may vary. | + | |
- | + | ||
- | === Node.js === | + | |
- | + | ||
- | < | + | |
- | $ emerge -av nodejs | + | |
- | </ | + | |
- | + | ||
- | Make sure the npm USE flag is set to get the node package manager as well | + | |
- | + | ||
- | ==== Cross Compiler Toolchain ==== | + | |
- | + | ||
- | At this time is wasn't possible yet to use a gentoo crossdev toolchain to compile | + | |
- | the firmware since it seems to require newlib-nano instead of the plain newlib gentoo | + | |
- | would like to merge. Aholler | + | |
- | + | ||
- | There wasn't enough time to hunt down this particular bug further so the | + | |
- | [[https:// | + | |
- | + | ||
- | + | ||
- | < | + | |
- | $ mkdir spark-core | + | |
- | $ cd spark-core | + | |
- | </ | + | |
- | + | ||
- | + | ||
- | < | + | |
- | $ wget https:// | + | |
- | $ tar xvjpf gcc-arm-none-eabi-4_8-2014q2-20140609-linux.tar.bz2 | + | |
- | </ | + | |
- | + | ||
- | Tell your system to pick it up automatically: | + | |
- | + | ||
- | < | + | |
- | + | ||
- | ==== Core/ | + | |
- | + | ||
- | < | + | |
- | $ git clone https:// | + | |
- | $ git clone https:// | + | |
- | $ git clone https:// | + | |
- | </ | + | |
- | + | ||
- | Make sure you checkout the three core repos into the same folder, otherwise it won't build. | + | |
- | + | ||
- | ==== Spark Server ==== | + | |
- | + | ||
- | < | + | |
- | $ git clone https:// | + | |
- | </ | + | |
- | + | ||
- | ==== Spark CLI ==== | + | |
- | + | ||
- | < | + | |
- | $ npm install -g spark-cli (need either root or sudo) | + | |
- | </ | + | |
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | ===== Configuration & Usage ===== | + | |
- | + | ||
- | + | ||
- | ==== Local Spark cloud server ==== | + | |
- | + | ||
- | < | + | |
- | $ cd spark-server/ | + | |
- | $ npm install | + | |
- | $ node main.js | + | |
- | + | ||
- | ------- | + | |
- | No users exist, you should create some users! | + | |
- | ------- | + | |
- | connect.multipart() will be removed in connect 3.0 | + | |
- | visit https:// | + | |
- | connect.limit() will be removed in connect 3.0 | + | |
- | Starting server, listening on 8080 | + | |
- | static class init! | + | |
- | core keys directory didn't exist, creating... / | + | |
- | Creating NEW server key | + | |
- | Loading server key from default_key.pem | + | |
- | set server key | + | |
- | server public key is: | + | |
- | -----BEGIN PUBLIC KEY----- | + | |
- | MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2UuRHTMfftLH/ | + | |
- | 5H3dFElK4J4Zp3SkF3Ere3pS/ | + | |
- | SESv5Ztt6zxlB4vbmzkK914bj/ | + | |
- | i+mdy/ | + | |
- | crAaE/ | + | |
- | 5eQ2T/ | + | |
- | nQIDAQAB | + | |
- | -----END PUBLIC KEY----- | + | |
- | + | ||
- | Your server IP address is: 192.168.1.100 | + | |
- | server started { host: ' | + | |
- | </ | + | |
- | + | ||
- | ==== Spark-CLI Configuration ==== | + | |
- | + | ||
- | Spark-CLI usually wants to connect to the official Spark cloud, so we need | + | |
- | to tell it where to connect instead. Since our spark-server told us it is | + | |
- | listening on **192.168.1.100** we just say: | + | |
- | + | ||
- | < | + | |
- | $ spark config local api-url http:// | + | |
- | $ spark config local | + | |
- | </ | + | |
- | + | ||
- | This will create/ | + | |
- | our wish to use the local cloud instance running on **192.168.1.100** instead of the public one. | + | |
- | At the point of writing these features are not yet documented but this | + | |
- | [[https:// | + | |
- | worked here. | + | |
- | + | ||
- | ==== Prepare a new Spark-Core ==== | + | |
- | + | ||
- | Let's leave the server running in this shell and open another terminal | + | |
- | to configure the core. Now it's also a good time to connect your Micro-USB cable to the Spark-Core. | + | |
- | + | ||
- | === Update factory firmware & CC3000 code === | + | |
- | + | ||
- | If this is a new Spark-Core, we want to run a special firmware program (deep-update), | + | |
- | + | ||
- | < | + | |
- | + | ||
- | < | + | |
- | FOUND DFU DEVICE 1d50:607f | + | |
- | checking file / | + | |
- | spawning dfu-util -d 1d50:607f -a 0 -i 0 -s 0x08005000: | + | |
- | dfu-util 0.7 | + | |
- | + | ||
- | [...] | + | |
- | + | ||
- | Filter on vendor = 0x1d50 product = 0x607f | + | |
- | Opening DFU capable USB device... ID 1d50:607f | + | |
- | Run-time device DFU version 011a | + | |
- | Found DFU: [1d50:607f] devnum=0, cfg=1, intf=0, alt=0, name=" | + | |
- | Claiming USB DFU Interface... | + | |
- | Setting Alternate Setting #0 ... | + | |
- | Determining device status: state = dfuERROR, status = 10 | + | |
- | dfuERROR, clearing status | + | |
- | Determining device status: state = dfuIDLE, status = 0 | + | |
- | dfuIDLE, continuing | + | |
- | DFU mode device DFU version 011a | + | |
- | Device returned transfer size 1024 | + | |
- | No valid DFU suffix signature | + | |
- | Warning: File has no DFU suffix | + | |
- | DfuSe interface name: " | + | |
- | Downloading to address = 0x08005000, size = 93636 | + | |
- | ............................................................................................ | + | |
- | File downloaded successfully | + | |
- | Transitioning to dfuMANIFEST state | + | |
- | Error during download get_status | + | |
- | Flashed! | + | |
- | </ | + | |
- | + | ||
- | Don't worry about the "Error during download get_status", | + | |
- | + | ||
- | < | + | |
- | + | ||
- | === ID your Core === | + | |
- | + | ||
- | Put your Spark-Core into Listening-Mode by pressing and holding the MODE button until the LED is blinking (2Hz) blue. Now run the following command to get your Core's ID: | + | |
- | + | ||
- | < | + | |
- | < | + | |
- | + | ||
- | Note/copy this ID, you'll need it soon again. | + | |
- | + | ||
- | === Upload local Spark-Cloud' | + | |
- | + | ||
- | Now that we've got the Spark-CLI ready, we can supply our Spark-Core with the public key | + | |
- | of our freshly deployed spark-server. This actually is a good practice against DNS hijacking or other | + | |
- | man-in-the-middle type attacks because our Spark-Cores can now determine that they are | + | |
- | communicating with our server and not someone else' | + | |
- | + | ||
- | < | + | |
- | $ cd spark-core/ | + | |
- | $ ls -al default* | + | |
- | </ | + | |
- | + | ||
- | < | + | |
- | -rw-r--r-- 1 chrono users 1679 Sep 25 11:27 default_key.pem | + | |
- | -rw-r--r-- 1 chrono users 451 Sep 25 11:27 default_key.pub.pem | + | |
- | </ | + | |
- | + | ||
- | Put the Spark-Core into DFU mode again and then execute: | + | |
- | + | ||
- | < | + | |
- | + | ||
- | < | + | |
- | Creating DER format file | + | |
- | running openssl rsa -in default_key.pub.pem -pubin -pubout -outform DER -out default_key.pub.der | + | |
- | checking file default_key.pub192_168_1_100.der | + | |
- | spawning dfu-util -d 1d50:607f -a 1 -i 0 -s 0x00001000 -D default_key.pub192_168_1_100.der | + | |
- | dfu-util 0.7 | + | |
- | + | ||
- | [...] | + | |
- | + | ||
- | Filter on vendor = 0x1d50 product = 0x607f | + | |
- | Opening DFU capable USB device... ID 1d50:607f | + | |
- | Run-time device DFU version 011a | + | |
- | Found DFU: [1d50:607f] devnum=0, cfg=1, intf=0, alt=1, name=" | + | |
- | Claiming USB DFU Interface... | + | |
- | Setting Alternate Setting #1 ... | + | |
- | Determining device status: state = dfuERROR, status = 10 | + | |
- | dfuERROR, clearing status | + | |
- | Determining device status: state = dfuIDLE, status = 0 | + | |
- | dfuIDLE, continuing | + | |
- | DFU mode device DFU version 011a | + | |
- | Device returned transfer size 1024 | + | |
- | No valid DFU suffix signature | + | |
- | Warning: File has no DFU suffix | + | |
- | DfuSe interface name: "SPI Flash : SST25x" | + | |
- | Downloading to address = 0x00001000, size = 1024 | + | |
- | . | + | |
- | File downloaded successfully | + | |
- | Okay! New keys in place, your core will not restart. | + | |
- | </ | + | |
- | + | ||
- | === Create new keys for your core === | + | |
- | + | ||
- | Put the Spark-Core into DFU mode again and then execute: | + | |
- | + | ||
- | < | + | |
- | $ mkdir core_keys | + | |
- | $ cd core_keys | + | |
- | $ spark keys doctor 1234567890abcdef | + | |
- | </ | + | |
- | + | ||
- | < | + | |
- | FOUND DFU DEVICE 1d50:607f | + | |
- | running openssl genrsa -out 1234567890abcdef_new.pem 1024 | + | |
- | running openssl rsa -in 1234567890abcdef_new.pem -pubout -out 1234567890abcdef_new.pub.pem | + | |
- | running openssl rsa -in 1234567890abcdef_new.pem -outform DER -out 1234567890abcdef_new.der | + | |
- | New Key Created! | + | |
- | FOUND DFU DEVICE 1d50:607f | + | |
- | FOUND DFU DEVICE 1d50:607f | + | |
- | running dfu-util -d 1d50:607f -a 1 -s 0x00002000: | + | |
- | running openssl rsa -in pre_1234567890abcdef_new.der -inform DER -pubout -out pre_1234567890abcdef_new.pub.pem | + | |
- | Saved! | + | |
- | checking file 1234567890abcdef_new.der | + | |
- | spawning dfu-util -d 1d50:607f -a 1 -i 0 -s 0x00002000: | + | |
- | dfu-util 0.7 | + | |
- | + | ||
- | Copyright 2005-2008 Weston Schmidt, Harald Welte and OpenMoko Inc. | + | |
- | Copyright 2010-2012 Tormod Volden and Stefan Schmidt | + | |
- | This program is Free Software and has ABSOLUTELY NO WARRANTY | + | |
- | Please report bugs to dfu-util@lists.gnumonks.org | + | |
- | + | ||
- | Filter on vendor = 0x1d50 product = 0x607f | + | |
- | Opening DFU capable USB device... ID 1d50:607f | + | |
- | Run-time device DFU version 011a | + | |
- | Found DFU: [1d50:607f] devnum=0, cfg=1, intf=0, alt=1, name=" | + | |
- | Claiming USB DFU Interface... | + | |
- | Setting Alternate Setting #1 ... | + | |
- | Determining device status: state = dfuUPLOAD-IDLE, | + | |
- | aborting previous incomplete transfer | + | |
- | Determining device status: state = dfuIDLE, status = 0 | + | |
- | dfuIDLE, continuing | + | |
- | DFU mode device DFU version 011a | + | |
- | Device returned transfer size 1024 | + | |
- | No valid DFU suffix signature | + | |
- | Warning: File has no DFU suffix | + | |
- | DfuSe interface name: "SPI Flash : SST25x" | + | |
- | Downloading to address = 0x00002000, size = 610 | + | |
- | . | + | |
- | File downloaded successfully | + | |
- | Transitioning to dfuMANIFEST state | + | |
- | Error during download get_status | + | |
- | Saved! | + | |
- | attempting to add a new public key for core 1234567890abcdef | + | |
- | submitting public key succeeded! | + | |
- | Okay! New keys in place, your core should restart. | + | |
- | </ | + | |
- | + | ||
- | + | ||
- | ==== Claim the Spark-Core ==== | + | |
- | + | ||
- | Press RESET on the core and/or hold the MODE button until the | + | |
- | core goes back to listening mode (blinking blue). In order to | + | |
- | create a new user and claim your first core we simply run: | + | |
- | + | ||
- | < | + | |
- | + | ||
- | < | + | |
- | If you run this the first time, you'll have to create an | + | |
- | account in your local cloud first: | + | |
- | + | ||
- | ======================================== | + | |
- | Setup your account | + | |
- | + | ||
- | Could I please have an email address? | + | |
- | and a password? | + | |
- | + | ||
- | Trying to login... | + | |
- | Login failed, Lets create a new account! | + | |
- | confirm password | + | |
- | creating user: myemail@domain.net | + | |
- | user creation succeeded! | + | |
- | Got an access token! 7yxc7yxc7yxc7yxc7yxc7yxc7yxc7yxc7yxc | + | |
- | Logged in! Saving access token: 7yxc7yxc7yxc7yxc7yxc7yxc7yxc7yxc7yxc | + | |
- | Using the setting " | + | |
- | + | ||
- | Now that we have created a user and are logged in (got an access token) | + | |
- | we can claim our core: | + | |
- | + | ||
- | ---------------------- | + | |
- | Finding your core id | + | |
- | + | ||
- | Your core id is: 1234567890abcdef | + | |
- | + | ||
- | ======================================== | + | |
- | Setup your wifi | + | |
- | + | ||
- | SSID: mySSID | + | |
- | Security 0=unsecured, | + | |
- | Wifi Password: myWifiPSK | + | |
- | Attempting to configure wifi on / | + | |
- | I said: w | + | |
- | Serial said: SSID: | + | |
- | I said: mySSID | + | |
- | + | ||
- | Serial said: mySSID | + | |
- | Security 0=unsecured, | + | |
- | I said: 3 | + | |
- | + | ||
- | Serial said: 3 | + | |
- | Password: | + | |
- | I said: myWifiPSK | + | |
- | + | ||
- | Serial said: myWifiPSK | + | |
- | Thanks! Wait about 7 seconds while I save those credentials... | + | |
- | + | ||
- | Awesome. Now we'll connect! | + | |
- | + | ||
- | If you see a pulsing cyan light, your Spark Core | + | |
- | has connected to the Cloud and is ready to go! | + | |
- | + | ||
- | If your LED flashes red or you encounter any other problems, | + | |
- | visit https:// | + | |
- | + | ||
- | Spark <3 you! | + | |
- | + | ||
- | Done! Your core should now restart. | + | |
- | + | ||
- | Please wait until your core is breathing cyan and then press ENTER | + | |
- | </ | + | |
- | + | ||
- | The Led should now slowly pulsate in cyan (blue+green) and your | + | |
- | spark-server should show something in the console like: | + | |
- | + | ||
- | < | + | |
- | | + | |
- | on ready { coreID: ' | + | |
- | ip: ' | + | |
- | product_id: 0, | + | |
- | firmware_version: | + | |
- | cache_key: ' | + | |
- | Core online! | + | |
- | </ | + | |
- | + | ||
- | === Check/List cores === | + | |
- | + | ||
- | < | + | |
- | $ spark list | + | |
- | </ | + | |
- | + | ||
- | < | + | |
- | Checking with the cloud... | + | |
- | Retrieving cores... (this might take a few seconds) | + | |
- | tinkertest (123456789abcdef) is online | + | |
- | Functions: | + | |
- | int digitalread(String args) | + | |
- | int digitalwrite(String args) | + | |
- | int analogread(String args) | + | |
- | int analogwrite(String args) | + | |
- | </ | + | |
- | + | ||
- | + | ||
- | ==== Compile firmware ==== | + | |
- | + | ||
- | < | + | |
- | $ cd core-firmware/ | + | |
- | </ | + | |
- | + | ||
- | + | ||
- | < | + | |
- | $ make clean all | + | |
- | </ | + | |
- | + | ||
- | < | + | |
- | Building core-common-lib | + | |
- | make[1]: Nothing to be done for `all' | + | |
- | + | ||
- | Building core-communication-lib | + | |
- | make[1]: Nothing to be done for `all' | + | |
- | + | ||
- | Building target: core-firmware.elf | + | |
- | Invoking: ARM GCC C++ Linker | + | |
- | mkdir -p ./ | + | |
- | arm-none-eabi-g++ -g3 -gdwarf-2 -Os -mcpu=cortex-m3 -mthumb | + | |
- | + | ||
- | Invoking: ARM GNU Create Flash Image | + | |
- | arm-none-eabi-objcopy -O binary core-firmware.elf core-firmware.bin | + | |
- | + | ||
- | Invoking: ARM GNU Create Flash Image | + | |
- | arm-none-eabi-objcopy -O ihex core-firmware.elf core-firmware.hex | + | |
- | + | ||
- | Invoking: ARM GNU Print Size | + | |
- | arm-none-eabi-size --format=berkeley core-firmware.elf | + | |
- | | + | |
- | 78672 | + | |
- | </ | + | |
- | + | ||
- | ==== OTA firmware update ==== | + | |
- | + | ||
- | < | + | |
- | $ spark flash 1234567890abcdef tinker | + | |
- | </ | + | |
- | + | ||
- | + | ||
- | < | + | |
- | Including: | + | |
- | / | + | |
- | attempting to flash firmware to your core 1234567890abcdef | + | |
- | flash core said {" | + | |
- | </ | + | |
- | + | ||
- | < | + | |
- | set_core_attributes { coreID: ' | + | |
- | userID: ' | + | |
- | FlashCore { coreID: ' | + | |
- | userID: ' | + | |
- | flash core started! - sending api event { coreID: ' | + | |
- | 192.168.1.100 - - [Sun, 05 Oct 2014 09:34:18 GMT] "PUT / | + | |
- | on response, no chunk, transfer done! | + | |
- | releasing flash ownership | + | |
- | flash core finished! - sending api event { coreID: ' | + | |
- | Connection from: 192.168.1.123, | + | |
- | on ready { coreID: ' | + | |
- | ip: ' | + | |
- | product_id: 0, | + | |
- | firmware_version: | + | |
- | cache_key: ' | + | |
- | Core online! | + | |
- | </ | + | |
- | + | ||
- | ==== Caveats ==== | + | |
- | + | ||
- | * Compiling through the local cloud is on the road map but doesn' | + | |
- | + | ||
- | {{tag>tag1 tag2 }} | + | |
- | + | ||
- | {{keywords> | + | |
~~DISCUSSION~~ | ~~DISCUSSION~~ | ||
- | |||
- | |||
- | When you are done with your draft, enter the following URL schema, create the page (with current date and your title) and copy the content into it: | ||
- | |||
- | https:// |