[Back to Home Page]

www.RomanBlack.com

"BlackNet" serial network protocol

A system for unlimited number of devices to transmit/receive serial data on 1 wire.
Originally written May 2009, added more diagrams and finally put on the net 27 Apr 2011.

What is it?
This is a system for many devices to be attached to one serial bus wire, where any devices can send/receive at
will with no conflicts. I designed this for use with my home automation controllers, but it has other uses like
multiple sensors or multiple controlled devices.

All information on this "BlackNet" page can be considered open source and used for whatever, but please
mention me if reproducing any of this work.

Hardware

The hardware side is very simple. One common ground. One "bus data" wire which is pulled up to 12v (or
13.8v) by a pull-up resistor. I ended up using a constant current pull-up of 150mA, so that I can use devices
powered from the bus wire itself. This means I only need a 2-wire cable for long runs like temperature sensors
on my solar panels etc, as they only draw a couple mA each they can use "ghost power" from the bus which
saves running power cables to these sensors. Most of my devices have 12v power cabling too so the ghost
power is just an option.

The BlackNet bus data line is both a transmit and receive line, for all devices.

Transmit. Each device can transmit by pulling the bus wire low to send one or more serial bytes. I used
standard 8N1 serial, 1 start bit, 8 data bits and 1 stop bit. If no device is transmitting, the bus wire remains high
at 12v.

Receive. Each device can receive any byte which is on the bus. So any device is capable of hearing what is
said by any other device.

Desired protocol system

What I wanted from this system;
Only 2 wires needed; bus and ground
No limit to number of devices*
Easy serial using UART or PIC bitbanging
No special hardware needed, just a PIC and a transistor
No limit on transmission distance or baudrate*
Ghost power option
Any device can listen if it chooses
Any device can transmit, or choose NOT to transmit
Transmitting device can transmit any number of bytes
If a device fails it doesn't affect other devices*
Devices can be added or disconnected at any time with no issues
Multiple masters (transmitters) with no issues
Multiple slaves (receivers) with no issues
Devices do not need serial receive capability or UART to work on the bus
All devices transmit in sequence
A failsafe way of identifying all transmitting devices
If any devices transmit garbage bytes it does not affect other devices

I should clarify some points;

There are no limits in the protocol to number of devices although some limitations will occur because of line
loading. It would still be reasonable to have hundreds of devices on the network if it has been set up for that.

The protocol itself has no limit on distance. For long distance networks the devices can just use a lower
baudrate.

A device that fails will not affect the network provided that it does not short circuit. Any other device failure
including; loss of power or open circuit or fail to transmit or garbage transmit will have no effect on other
devices.

No device is REQUIRED to transmit. This is important, it means any device can transit only if it chooses to,
without affecting the chain of other devices sending/receiving. This is good for devices sending an infrequent
alarm or sensors that only need to "report in" after long intervals. It also means that if any device fails or is
disconnected there is no effect on the other devices. Obviously this is also good for devices that are in low-
power mode most of the time.

All devices CAN listen, but no device is REQUIRED to listen. This is a big strength of the protocol. It means
that a device that sends data on the bus is not required to have a UART to receive bytes from other devices. All
it needs is to monitor that the bus is free for X amount of time.

Any device can be a transmitter or a receiver or combination of both. This is designed so that some devices
might be light switches (transmit), some might be lights (receive) some might do both tasks. There can be
many controllers on the bus controlling the one receiving slave or vice versa. The system is designed to be very
open-ended.

The protocol
I won't claim any great originality here, over the years I have seen many systems for a number of devices on
one serial wire so I just copied different elements from existing systems to combine together to create a very
open ended and simple 1-wire network to suit my needs.

It is basic and sequential ie a "round robin"; No device can transmit until device1 transmits. Then device2
can transmit IF it chooses, as many bytes as it chooses. Then device 3 can transmit IF it chooses, etc.
Every device gets a chance to talk, then that "loop" is over.

For my home automation I chose to make device1 the master clock, so every 1/2 second it transmits the full
date/time etc. So devices can only transmit once per 1/2 second. The protocol does not require any specific
loop timing, it is only required that every device has had its CHANCE to talk, then device1 talks again to start
the loop all over again.

Making sure all devices talk in turn! Each device knows its own number. After device1 finishes talking, the
bus goes free again. After the bus has been free for a period of T*2 then device 2 is allowed to start talking (if it
chooses). Then after the bus has been free for a time of T*3 then device3 is allowed to start talking. Then after
the bus is free for a time of T*4 device4 is allowed to talk and so on.

If any device chooses NOT to talk, then one time period T later the next device can talk so there is no penalty
in bus time. This is very efficient for devices like light switches etc that only need to talk when someone
presses the switch.

Yes this can become slow with a very large number of devices IF they all need to talk at once, as the total bus
loop time is an exponential function of number of talking devices;

The total number of wait periods (T) for any number of devices (n) is roughly given by the formula;
total_T = n * (n / 2)
So for 10 devices the total T time needed will be 50 T.
For 100 devices the total T time needed will be 5000 T.
(However this is only true if ALL 100 devices talk in that cycle. In a best case cycle where none of those 100
devices talk the total cycle time is only 100 T.)

As the time period T is only a few uS this is still fine for 100+ devices to all talk in a 0.5 second bus loop time.
The protocol is based on minimal hardware and open-endedness, speed comes last. The open-ended nature of
the protocol allows any or all devices to NOT talk so there is an option for very large numbers of devices on the
bus if they don't all need to talk in every loop. Obviously, there is no time used up by devices that only listen!

About time period T and when a device knows it's turn to talk
Originally I was going to use a large time period for T that is longer than the total time for one serial byte, so
that some high bits in a byte are not mistaken for a finished transmission.

However it only requires ONE safe period this length after each transmission to ensure transmission has
finished, then after that the period T can be shorter than a serial byte. A period T of 20uS is fine even with
cheap PICs running from 4MHz xtal (1uS timing). The single safe period should be longer than a serial byte, for
example can be 200uS if using 56kbaud serial.

Procedure for a cheap transmitting device detecting time of 7T;

1. first, wait for any device talking (bus low)
2. wait for bus going low-high, then reset timer
3. if bus goes low again go back to 2
4. if timer reaches safeperiod + 7T, start transmission!

Practical uses
Obviously this is not a good system for devices that need to send lots of dense data. However it would be a
good system where the devices need to be cheap as the devices do not need UART to be able to talk on the
network. They only need to sense the TIME the bus is free, then transmit a data packet using bit-banged
serial.

This could be useful for a very large number of cheap sensors on a 2 wire bus, like solar panel sensors that
only need to talk occasionally to signal that each panel is working, or signal a panel fail condition.

It can be ideal to network cheap sensors within a building, like hundreds of light switches (or people sensors)
where they only need to talk if a switch or sensor is activated. The same network can have thousands of
receiving devices (like smart lights) that can just listen to the bus to see if they are commanded on or off.

It might be useful for industrial or alarm sensors, especially as all devices are in parallel there is no problem
attaching or detaching devices to/from the bus at any time while the bus is running.

Practical implementation for my own network

I decided to make device1 be a system clock. This transmits a time code of "year-month-day-HH-MM-SS"
every 1/2 a second, and always starts a bus cycle every 1/2 a second.

All devices afer device1 can know the time (if they receive bus data) and this allows for obvious time related
features where devices can switch on/off at specific times without those devices needing to talk on the bus.

Another benefit is that devices can talk only on specific cycles, so a device that only needs to talk every 30
seconds (like a room temperature sensor) can do so and uses very little bus time in total.

Another simple trick I added later was to make every device transmit its device number as the first byte
(assuming it is a talking device). This is not required by the protocol but seemed useful for a number of
reasons.

For hardware I used a constant current bus pullup of 150mA, to 12v. The 12v is unregulated battery voltage
generally 11v to 13.8v range. I have not used ghost power in any devices so far, as I used telephone style cable
and run these 4 wires;
+12v unreg
+9v reg
Data bus (pullup +12v 150mA)
Ground

As the bus data is normally high (same polarity as a PIC USART), it can be connected directly into a PIC
USART RX pin through a 47k resistor. This allows minimal hardware receive with only one resistor (uses the
PIC internal port pin diode).

Output to the bus is through a simple NPN transistor that is capable of pulling the 150mA bus line low. That
means pretty much any cheap small-signal NPN like a BC337. If the serial output is bit-banged this only
requires one resistor and transistor, and the bit banged serial from the PIC output pin must be inverted
compared to a normal PIC USART TX pin.

However if the output is from the PIC USART TX pin a second transistor is needed for data inversion. I figured
that for minimal devices they can bit bang the output, and more complex devices can afford another 5 cent
transistor. The benefit is that the RX does not need inversion, and the bus line is normally high allowing ghost
power.

Problems and limitations

Obviously this network is for small data packets, a handful of bytes per device. Although any talking device can
transmit any number of bytes generally they should be transmitting a small amount of data.

As it is a "round robin" system (taking turns), devices need to know their own device number and have
individual device numbers. This was fine for my needs and should be ok for permanent installations but could
be a big limitation in some systems. It should be possible for a master to track all device numbers in use, and
identify new devices and assign them a number although I didn't bother with auto numbering and just manually
assigned numbers to each node I built.

There is a problem if a device fails short circuit as they are parallel on the bus the entire bus goes down.
However this type of fault applies to most types of network busses and is easy to trace. A device that fails
open circuit will not affect the bus.

There is a time problem if there are a lot of devices that must talk during one cycle. My home system with 0.5
seconds per cycle is limited to about 100 devices talking at once. However most devices do not need to talk
EVERY cycle, and bus time can easily be distributed this way. There are plenty of applications that can have a
large number of talking or listening devices without all the devices needing to talk at once.

A big speed improvement

Because I made the talking devices transmit their number, this allows a "fast mode" where device8 does not
have to wait the typical 8T time before talking. If it listens and hears device7 has just finished talking then
device8 can talk immediately (as it knows it is next).
Like most of the protocol this "fast mode" is open ended and any device may use fast mode or not without the
entire network needing to be structured this way. It would be most useful for higher numbered devices that
might be clumped in groups (to talk in groups). Of course if there are only 20 or 30 must-talk devices on a
network then fast mode is really not needed.

Ghost power
Because the bus line spends most of its time high (especially when there is a lot of devices!) it is ideal to
provide ghost power to small low-power devices.

The bus would charge caps in all devices and provide enough power to run the devices. With low power PICs
and low quiescent current regulators common and cheap now it is conceivable for a simple 2-wire bus to power
a large number of devices as well as carrying data.

A practical ghost power circuit in a device would have a diode and resistor charging a large cap to approx the
bus high voltage (12v). The resistor is needed to reduce peak currents after any 0 bit. As the data 0 and 1 bits
are detected at logic levels (approx 1v and 3.5v) the data on the bus will not be affected by the ghost power
charging caps etc which will ideally occur in the 9v to 12v range. The cap in each device should be large
enough so that it's voltage does not drop much below 9v when the bus is transmitting data.

Ideally a 5v (or 3.3v) low quiescent curent low dropout voltage regulator would be used. If so, a device might use
less than 1mA average current from the bus so it would be possible to have a quite a lot of ghost powered
devices on a network. Although I like the idea of ghost power I did not use it on my internal 4-wire home
network. I may explore ghost power devices with an external 2-wire network at a later date.

My home network
I won't go into the exact details of my network setup as it performs some security tasks but here is a glimpse
at the PCB I whipped up in 2009 for my BlackNet master clock (device1).

At the right you can see the 4 wires to the network system, and in the circled area you can see that the serial
output system just uses 2 SMD transistors and a few resistors.

The master clock also has a multi-digit display and a Dallas Semi 32kHz temperature compensated precision
oscillator module to give extremely accurate timekeeping. It is accurate to 2PPM which is around 1 minute a
year error! I also have a GPS module that I will eventually connect as another node on the network that will act
as a second "clock" and send error data to the master clock and allow the master clock to gently adjust it's
time to re-synchronise to match the GPS clock.

Minimum parts count - multiple devices

(minor update 30th Apr 2011 - in response to a question I was asked about minimal hardware applications).

For an extremely low parts count solution you can connect multiple microcontrollers on the one data bus line
and use a 5v data bus. This uses a simple 270 ohm pull-up resistor for the entire data bus.

This method can be used for multiple PICs on a single PCB or within a single device (like a large display unit).

All that is needed is one diode per PIC microcontroller if you are using the USART TX output. If you are bit-
banging serial output on low cost PICs (like a multiple sensor array) then the diode is not needed as the
firmware can toggle the serial output between pull-down and high impedance states.

This simple hardware will still allow hundreds of devices on the one data bus line, although the 5v line is not
suitable for ghost power and will not generally allow the long distances obtainable from a 12v line and constant
current pull-up.

- end -

[Back to Home Page]