/SEM-Bootloader

SEM TFTP bootloader for Arduino Ethernet (incl. support for Atmega 1284P and Wiznet W5200, W5500)

Primary LanguageArduino

TFTP Bootloader for ATmega328P/Atmega2560/Atmega1284P

/w support for Wiznet5100, W5200 and W5500

This is a beta stage bootloader for Arduino Ethernet board and the regular Arduino with Ethernet Shield. It is based on previous unfinished work by the Arduino developers. The bootloader implements a TFTP server on the Arduino board and flashing works using any regular TFTP client.

The Files and Folders in this Repository

The structure of this repository is made to follow the standarts of the Arduino IDE. This way you can simply copy the folders in your sketchbook and be ready

  • hardware: This is where the bootloader resides.
  • java-client: Demo client for the bootloader. Inherited by the initial project. Untested and probably non-functional
  • libraries: Helper libraries to support functions of the bootloader
  • utilities: Various stuff used for development and debugging

Downloading and Installing Files

First of all, you need to clone or download the repository. To clone the repository you need to have git installed, then you can run git clone https://github.com/arjenhiemstra/SEM-Bootloader.git in a directory. This way you can later update your local repository by running git pull inside the SEM-Bootloader directory.

In case you want to avoid this hassle, you can use the ZIP button at the top of the page to download the latest snapshot of the repository in a zip archive and extract it.

After that you have to copy the hardware and libraries folders inside your sketchbook folder. Take extra care during coping not to overwrite any other files. Restart the Arduino IDE to load the new boards and libraries.

Installing the Bootloader

To burn the bootloader, you will need an ISP(in-system programmer) like [AVR-ISP][2], [USBtinyISP][3] or you can build a [ParallelProgrammer][4] or an [ArduinoISP][5]. The first three programmers should be connected to the ICSP pins (the 2 by 3 pin header) and make sure you plug it in the right way. The board must be powered by an external power supply or the USB port. In the case of ArduinoISP you should consult the above link for further instructions on how to build and use. [2]: http://www.atmel.com/dyn/products/tools_card.asp?tool_id=2726 [3]: http://www.ladyada.net/make/usbtinyisp/ [4]: http://arduino.cc/en/Hacking/ParallelProgrammer [5]: http://arduino.cc/en/Tutorial/ArduinoISP

After you have connected the Arduino board and the programmer to your computer launch the Arduino IDE. Navigate to the Tools > Board menu and select SEMboot Atmega <chip> (<Wiznet Chip>) which reflects your Arduino/Atmega board and wiznet chip combination. Then go to Tools > Programmer and select the programmer you are using. In case you are using ArduinoISP, make sure that the selected port in the Tools > Serial Port menu refers to the ArduinoISP and not the board that you want to burn the bootloader on. Now, just launch the Tools > Burn Bootloader command and wait for about 15 seconds for the operation to complete.

In the case of Arduino Mega the previous ISPs do not reliably work because of design differences. To install the bootloader in these boards you will need one of the more expensive models, like [Atmel's AVRISP mkII][6], [Olimex's AVR-ISP-MK2][7] or another similar programmer. If you don't have any compatible programmer but more than one Arduinos available, you can use Nick Gammon's excellent [guide][8] on how to use one to program the other. This method has been reported to work for flashing Ariadne on Arduino Mega. [6]: http://www.atmel.com/tools/AVRISPMKII.aspx [7]: https://www.olimex.com/Products/AVR/Programmers/AVR-ISP-MK2/ [8]: http://www.gammon.com.au/forum/?id=11635

Serial Flashing

Ariadne bootloader supports flashing through serial like any other regular bootloader. This way of uploading is built upon the default bootloaders so it should be pretty straight forward to use. Just plug in the USB cable and select the serial port and the appropriate board from the Tools > Board menu. After that you must press the reset button and the indication LED on pin 13 or pin 9, in case of Arduino Ethernet, will start blinking rapidly. This means that the bootloader is running and the Arduino is ready to be programmed. If there is a valid program already flashed on the Arduino, you have to reprogram the device in the next 5 seconds. If you don't, the bootloader will initiate the program that is already in the Arduino. In case there is no program flashed or the program has been marked as invalid, the bootloader will never time out and you can reprogram it at any time.

After a succesful flashing,

  • Arduino Duemilanove will automatically start the user's application.
  • Arduino Uno will do a reset cycle and start the program after the bootloader times out.
  • Arduino Mega will behave in a similar way as Arduino Uno

This happens because Uno has the autoreset feature that resets the board after a serial connection.

Due to the fact that "autoreset" for remote tftp programming is implemented using a watchdog timer timeout, the bootloader will do a full cycle after every reset, physical or software. For those who miss the Adaboot No-Wait Mod-like functionality, we have been testing some options on how to circumvent these limitations, but they still need refinement.

Default Network Settings

The default built-in network settings of the bootloader are listed below.

`

  • IP Address: 192.168.84.128

  • Subnet Mask: 255.255.255.0

  • Gateway: 192.168.84.1

  • MAC Address: 0xDE.0xAD.0xBE.0xEF.0xFE.0xED

  • TFTP Negotiation Port: 69

  • TFTP Data Port: 46969 `

NOTE: The Arduino Ethernet that were sent as Indiegogo Perks were flashed with different network settings. Their bootloader still defaults to the previous settings but the current values are on the accompaning leaflet.

Configuring Network Settings

These can be changed using our NetEEPROM library. The library is going to have it's own documentation on how it can be used but for the purpose of changing and reading the network settings you can use the included examples. To load them navigate to File > Examples > NetEEPROM and select one of the examples. You can write the network settings using the WriteNetworkSettings sketch or print them to the serial using the ReadNetworkSettings.

Note that the settings array in the WriteNetworkSettings sketch hasn't got the settings in the usual order but rather in the order that W5100 reads them, so make sure you have put the correct values. If you set the network settings you also have to set the TFTP data transfer port. The default is good enough but you may need to change it for the reasons that are listed below in the [Configuring your Router for Remote Flashing][9] section. There is also documentation on how use these sketches in the form of comments so be sure to read them. [9]: https://github.com/codebendercc/Ariadne-Bootloader#configuring-your-router-for-remote-flashing

TFTP Flashing

Now for the real reason we made this bootloader and why you should use it. First of all you can watch Ariadne in action in this how-to video for remote flashing using TFTP [here][10]. In the video you may notice that the board is being reset by hand. In the next couple of weeks we are going to release the library that will allow remote resetting through a simple web server with some security measures. More on that as the library progresses. [10]: http://youtu.be/KCHqhV6xPMg

Converting your sketch to the right format

Unlike serial flashing that uses HEX files to flash the Arduino, the TFTP server implemented in the bootloader works with binary files. This means that you have to manually convert your programs to the right format. To do that, first build your sketch inside Arduino IDE using the Verify button. After that, without exiting the Arduino IDE you need to navigate to the temporary directory where your project was built. On all platforms, you can find out the temporary build folder by checking Show verbose output during compilation in Arduino IDE's preferences dialog and the path for the compiled .hex file will be printed in the last line of the compilation output.

Windows

On Windows, the temporary directory can be

C:\Documents and Settings\Administrator\Local Settings\Temp\build5571819468326525374.tmp\Blink.cpp.hex
"C:\Program Files\Arduino\hardware\tools\avr\bin\avr-objcopy.exe" -I ihex
"C:\Documents and Settings\Administrator\Local Settings\Temp\build5571819468326525374.tmp\Blink.cpp.hex"
-O binary Blink.cpp.bin

Linux

OS X

That is C:\Users\owner\AppData\Local\Temp\ on Windows, /tmp on Linux. On MacOS you'll need to go to Arduino's preferences, and check the "Show verbose output during compilation" checkbox. After that, when you compile, you will see the path for the compiled .hex file in the last line of the compilation output.. There you will find a folder named something like build2429295348207572157.tmp. That is where the Arduino temporary build files reside. Enter the directory and make sure that there is a .elf or a .hex file with the same name as your sketch. That is the file you need to convert. To achieve that you have to run one of the following commands in a terminal.

  • avr-objcopy -I ihex [sketch].hex -O binary [sketch].bin

In Windows and MacOS you can find the avr-objcopy program in your Arduino IDE installation folder under hardware/tools/avr/bin/. Or,if you have scons installed, you can use the modified SConstruct script you can find in SEM-Bootloader/utilities. This being based on the arscons script, it can be used in two ways. If you used the previous process to generate the HEX file you can just copy the SConstruct file inside the temporary Arduino IDE build directory (as mentioned above) and run scons in a terminal inside that directory.

The other way to use it is to copy the SConstruct script inside the sketch's directory and, as above, run scons in a terminal inside that directory. This way you will build your project outside Arduino IDE creating the .bin file in the process. Note that this way the sketch's folder will be polluted with Arduino's build files, much like the temporary directory Arduino IDE uses.

For testing purposes you can find a blink sketch in binary form inside the SEM-Bootloader/utilities/tests/blink folder. The fade sketch in the tests/fade folder will also give you a view of what a failed upload looks like. This sketch fails because it is written in plain C and not in Arduino. That way it lacks some "signatures" the bootloader uses to validate Arduino sketches. The third sketch in tests/led_display is an easter egg for which you need to find out how we had our led matrices connected on Arduino Uno. Or we might release the schematics at some point. Who knows.

Using a tftp client to upload the sketch

Now that the binary is ready, you have to upload it. First you have to connect to your Arduino using any tftp client you may have on your computer. All three major operating systems have their own clients that you can use through the command line.

Windows

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Administrator>tftp

Transfers files to and from a remote computer running the TFTP service.

TFTP [-i] host [GET | PUT] source [destination]

  -i              Specifies binary image transfer mode (also called
                  octet). In binary image mode the file is moved
                  literally, byte by byte. Use this mode when
                  transferring binary files.
  host            Specifies the local or remote host.
  GET             Transfers the file destination on the remote host to
                  the file source on the local host.
  PUT             Transfers the file source on the local host to
                  the file destination on the remote host.
  source          Specifies the file to transfer.
  destination     Specifies where to transfer the file.


C:\Documents and Settings\Administrator>tftp -i 192.168.84.128 PUT sketch.bin
tftp: can't read from local file 'sketch.bin'

C:\Documents and Settings\Administrator>tftp -i 192.168.84.128 PUT sketch.bin

Linux

On some Linux distributions, like Fedora/RedHat, before you use tftp, you should load the ip_conntrack_tftp module or the tftp client won't be able to acknowledge(ACK) the incoming packets. That is needed because TFTP is insecure and it is not enabled by default. Other distributions like Arch, don't need this step. To do that, open a terminal and run

modprobe ip_conntrack_tftp

as root using su or sudo.

After that open a terminal as a regular user and type

tftp [ip] [port]

For the default bootloader settings that would be:

tftp 192.168.84.128 69

In this case it could just be tftp 192.168.84.128 as 69 is the default tftp port and the client would automatically connect to it. For any other port you have to explicitly set it.

Now you should have been greeted by the

tftp>

prompt.

First you have to enter this command:

tftp> mode octet

This way you tell the TFTP client to send binary data. This is absolutely needed as if your client is in netascii mode, uploading will fail. After this, it is advised to use the two following commands to make the process more informative so you can have a better view of what is happening, but they are not needed.

tftp> trace
tftp> verbose

Now to actually upload the binary file all you have to do is reset the board and in the next 5 seconds run the following command.

tftp> put [sketch].bin

The 5 second time frame is in case you already have a valid program uploaded on the Arduino. In case you don't have a program loaded or it has been marked invalid, you don't have any time constraints.

Now you should see your tftp client sending packets and the indication LED on pin 13 or pin 9 blinking in a random way, almost like having a hiccup. A correct output sample of the TFTP client uploading the blink sketch is below:

tftp> mode octet
tftp> trace
Trace mode on.
tftp> verbose
Verbose mode on.
tftp> put blink.bin
sent WRQ <file: blink.bin, mode: octet <>>
received ACK <block: 0>
sent DATA <block: 1, size: 512>
received ACK <block: 1>
sent DATA <block: 2, size: 512>
received ACK <block: 2>
sent DATA <block: 3, size: 512>
received ACK <block: 3>
sent DATA <block: 4, size: 512>
received ACK <block: 4>
sent DATA <block: 5, size: 42>
received ACK <block: 5>
tftp>

After a successful upload the bootloader will start the uploaded application instantly.

In case that for some reason the upload fails, first of all stop your TFTP client from sending any more packets. After that you should wait for the upload process on the bootloader to timeout. That takes about 5 seconds too. For this period you should witness the indication led doing some random blinking. After the timeout and since there is no valid program in the memory, the TFTP server should restart itself and wait for a new upload.

Configuring your Router for Remote Flashing

If you are having trouble flashing your Arduino at home from the road, you probably need to enable [port forwarding][11]. You need to forward ports 69 and 46969 to your Arduino in your router's configuration. In case you have changed the incoming data port from 46969 to another port i.e. 50232, you have to forward 50232 port instead of 46969. This is particularly useful when you have more than one Arduinos, that you want to flash, behind your router. In addition to this you are going to have to translate an external port of your choice on the router to the internal port and ip of the Arduino in the local network. [11]: http://en.wikipedia.org/wiki/Port_forwarding

An example is that you have 2 devices, one at 192.168.84.128 and one at 192.168.84.129. They both listen to port 69 for the initial connection. In this case you can translate external port 6969(any random port will do) on your router to 192.168.84.128:69 and external port 6970 to 192.168.84.129:69 and specify these in the tftp client you are using.

Port Forward has [excellent guides][12] on how to enable port forwarding for a vast number of routers. [12]: http://portforward.com/english/routers/port_forwarding/

Codebender

One of the best and easiest ways to use this bootloader is along with [codebender.cc][13]. Just register, enter your Arduino's IP (external IP for those in corporate or home networks behind NAT) and flash. [13]: http://codebender.cc

Helper Libraries

Supported Boards

Supported MCUs:

  • ATmega328
  • ATmega2560
  • Atmega1284P

Supported ethernet controllers:

  • WizNet W5100
  • WizNet W5200
  • WizNet W5500

Tested Arduino Boards:

Other Tested Boards

These boards have been also tested and found to be working with Ariadne Bootloader. I don't have any of these so a big thank you goes to the people that took their time to test them.

Contributors

Hachi Manzur (AVRISP mkII programmer, testing)

Over the time I will add more people who have helped in this list. If you feel that you should be in this list, please contact me via email.

Roadmap (outdated)

Right now the main focus for the first packaged release is bug fixing and improve existing functionality. That is why we encourage you to use the bootloader and report any bugs, misbehaviours or feature requests here on github. There is also on going work to work on the Arduino Mega and support for Arduino Leonardo is planned after that. Support for other ethernet or wifi controllers is being discussed but after the bootloader has been stabilized.

Acknoledgements

SEM-Bootloader bootloader is built upon some great open source projects.

Credit should also go to mharizanov for commenting some of the initial Arduino code, making it easy for me to start and follower who's sketches served as a starting point for the included NetEEPROM and EthernetReset libraries.

License

This is free software and it is released under the [GPLv2, GNU General Public License][99] [99]: https://www.gnu.org/licenses/gpl-2.0.html