Pure Python Implementation

This sections explains the python implementation of the IsoTP protocol.

Transport layer

Depending on your constraints, you may want to have the IsoTP protocol layer to run in Python (in the user space). For example, if you want to rely on python-can for the support of your CAN interface, you will need to run the IsoTP layer in Python.

In such case, the isotp.TransportLayer will be the right tool. One must first define functions to access the hardware and provide them to the isotp.TransportLayer as parameters named rxfn and txfn.

class isotp.TransportLayer(rxfn, txfn, address, error_handler=None, params=None, read_timeout=0.05)[source]

An IsoTP transport layer implementation that runs in a separate thread. The main public interface are start, stop, send, recv. For backward compatibility, this class will behave similarly as a V1 TransportLayer if start/stop are never called; meaning that process() can be called by the user

Parameters:

rxfn (Callable : expected signature: my_rxfn(timeout:float) -> Optional[isotp.CanMessage]) – Function to be called by the transport layer to read the CAN layer. Must return a isotp.CanMessage or None if no message has been received. For optimal performance, this function should perform a blocking read that waits on IO
txfn (Callable : expected signature: my_txfn(msg:isotp.CanMessage) -> None) – Function to be called by the transport layer to send a message on the CAN layer. This function should receive a isotp.CanMessage
address (isotp.Address) – The address information of CAN messages. Includes the addressing mode, txid/rxid, source/target address and address extension. See isotp.Address for more details.
error_handler (Callable) – A function to be called when an error has been detected. An isotp.IsoTpError (inheriting Exception class) will be given as sole parameter. See the Error section When started, the error handler will be called from a different thread than the user thread, make sure to consider thread safety if the error handler is more complex than a log.
params (dict) – Dict of parameters for the transport layer. See the list of parameters
read_timeout (float) – Default blocking read timeout passed down to the rxfn. Affects only the reading thread time granularity which can affect timing performance. A value between 20ms-500ms should generally be good. MEaningless if the provided rxfn ignores its timeout parameter

If python-can must be used as CAN layer, one can use the isotp.CanStack and isotp.NotifierBasedCanStack which extends the TransportLayer object with predefined functions that calls python-can.

class isotp.NotifierBasedCanStack(bus, notifier, *args, **kwargs)[source]

The IsoTP transport layer pre configured to use python-can as CAN layer and reading through a can.Notifier. python-can must be installed in order to use this class. All parameters except the bus and the notifier parameter will be given to the TransportLayer constructor

This class reads by registering a listener to the given notifier and sends by calling bus.recv.

Parameters:

bus (can.BusABC) – A python-can Bus object implementing send used for transmission
notifier (can.Notifier) – A python-can Notifier object onto which a new listener will be added
args (N/A) – Passed down to TransportLayer. rxfn and txfn are predefined
kwargs (N/A) – Passed down to TransportLayer. rxfn and txfn are predefined

class isotp.CanStack(bus, *args, **kwargs)[source]

The IsoTP transport layer pre configured to use python-can as CAN layer. python-can must be installed in order to use this class. All parameters except the bus parameter will be given to the TransportLayer constructor

This class directly calls bus.recv, consuming the message from the receive queue, potentially starving other application. Consider using the NotifierBasedCanStack to avoid starvation issues

Parameters:

bus (can.BusABC) – A python-can bus object implementing recv and send
args (N/A) – Passed down to TransportLayer. rxfn and txfn are predefined
kwargs (N/A) – Passed down to TransportLayer. rxfn and txfn are predefined

Note

The CanStack exists mainly for backward compatibility with v1.x. It is suggested to use the NotifierBasedCanStack to avoid starvation issues and achieve better performances.

The CAN messages going in and out from the transport layer are defined with isotp.CanMessage.

class isotp.CanMessage(arbitration_id=0, dlc=0, data=b'', extended_id=False, is_fd=False, bitrate_switch=False)[source]

Represent a CAN message (ISO-11898)

Parameters:

arbitration_id (int) – The CAN arbitration ID. Must be a 11 bits value or a 29 bits value if extended_id is True
dlc (int) – The Data Length Code representing the number of bytes in the data field
data (bytearray) – The 8 bytes payload of the message
extended_id (bool) – When True, the arbitration ID stands on 29 bits. 11 bits when False
is_fd (bool) – When True, message has to be transmitted or has been received in a CAN FD frame. CAN frame when set to False
bitrate_switch (bool) – When True, message has the Bit Rate Switch (BRS) bit set.

Parameters

The transport layer params parameter must be a dictionary with the following keys.

stmin (int)

default: 0

The single-byte Separation Time to include in the flow control message that the layer will send when receiving data. Refer to ISO-15765-2 for specific values. From 1 to 127, represents milliseconds. From 0xF1 to 0xF9, represents hundreds of microseconds (100us, 200us, …, 900us). 0 Means no timing requirements

blocksize (int)

default: 8

The single-byte Block Size to include in the flow control message that the layer will send when receiving data. Represents the number of consecutive frames that a sender should send before expecting the layer to send a flow control message. 0 means infinitely large block size (implying no flow control message)

tx_data_length (int)

default: 8

The maximum number of bytes that the Link Layer (CAN layer) can transport. In other words, the biggest number of data bytes possible in a single CAN message. Valid values are : 8, 12, 16, 20, 24, 32, 48, 64.

Large IsoTP frames will be transmitted in small CAN messages of this size except for the last CAN message that will be as small as possible, unless padding is used.

tx_data_min_length (int)

default: None

Sets the minimum length of CAN messages. Message with less data than this value will be padded using tx_padding byte or 0xCC if tx_padding=None.

When set to None, CAN messages will be as small as possible unless tx_data_length=8 and tx_padding != None; in that case, all CAN messages will be padded up to 8 bytes to be compliant with ISO-15765.

Valid values are : 1, 2, 3, 4, 5, 6, 7, 8, 12, 16, 20, 24, 32, 48, 64.

override_receiver_stmin (float or None): default: None

Time in seconds to wait between consecutive frames when transmitting. When set, this value will override the receiver stmin requirement. When None, the receiver stmin parameter will be respected. This parameter can be useful to speed up a transmission by setting a value of 0 (send as fast as possible) on a system that has low execution priority or coarse thread resolution.

This parameter replace the squash_stmin_requirement parameter available in v1.x

rx_flowcontrol_timeout (int)

default: 1000

The number of milliseconds to wait for a flow control frame before stopping reception and triggering a FlowControlTimeoutError. Defined as N_BS bs ISO-15765-2

rx_consecutive_frame_timeout (int)

default: 1000

The number of milliseconds to wait for a consecutive frame before stopping reception and triggering a ConsecutiveFrameTimeoutError. Defined as N_CS by ISO-15765-2

tx_padding (int or None)

default: None

When not None represents the byte used for padding messages sent. No padding applied when None unless tx_data_min_length is set or CAN FD mandatory padding is required.

wftmax (int)

default: 0

The single-byte “Wait Frame Max” to include in the flow control message that the layer will send when receiving data. When this limits is reached, reception will stop and trigger a MaximumWaitFrameReachedError

A value of 0 means that wait frames are not supported and none shall be sent.

max_frame_size (int)

default: 4095

The maximum frame length that the stack will accept to receive. ISO-15765-2:2016 allows frames as long as 2^32-1 (4294967295 bytes). When a FirstFrame is sent with a length longer than max_frame_size, the message will be ignored, a FlowControl message with FlowStatus=2 (Overflow) will be sent and a FrameTooLongError will be triggered.

This parameter mainly is a protection to avoid crashes due to lack of memory (caused by an external device).

can_fd (bool)

default: False

When set to True, transmitted messages will be CAN FD. CAN 2.0 when False.

Setting this parameter to True does not change the behavior of the TransportLayer except that outputted message will have their is_fd property set to True. This parameter is just a convenience to integrate more easily with python-can

bitrate_switch (bool)

default: False

When set to True, tx message will have a flag bitrate_switch marked as True, meaning that the underlying layer shall perform a CAN FD bitrate switch after arbitration phase.

Setting this parameter to True does not change the behavior of the TransportLayer except that outputted message will have their bitrate_switch property set to True. This parameter is just a convenience to integrate more easily with python-can

default_target_address_type (int)

default: Physical (0)

When using the TransportLayer.send method without specifying target_address_type, the value in this field will be used. The purpose of this parameter is to easily switch the address type if an application is not calling send() directly; for example, if you use a library that interact with the TransportLayer object (such as a UDS client).

Can either be Physical (0) or Functional (1)

rate_limit_enable (bool)

default: False

Enable or disable the rate limiter. When disabled, no throttling is done on the output rate. When enabled, extra wait states are added in between CAN message transmission to meet rate_limit_max_bitrate

Refer to Rate Limiter Section for more details

rate_limit_max_bitrate (int)

default: 10000000 b/s

Defines the target bitrate in Bits/seconds that the TransportLayer object should try to respect. This rate limiter only apply to the data of the output messages.

Refer to Rate Limiter Section for more details

rate_limit_window_size (float)

default: 0.2 sec

Time window used to compute the rate limit. The rate limiter algorithm works with a sliding time window. This parameter defines the width of the window. The rate limiter ensure that no more than N bits is sent within the moving window where N=(rate_limit_max_bitrate*rate_limit_window_size).

This value should be at least 50 msec for reliable behavior.

Refer to Rate Limiter Section for more details

listen_mode (bool)

default: False

When Listen Mode is enabled, the TransportLayer will correctly receive and transmit ISO-TP Frame, but will not send Flow Control message when receiving a frame. This mode of operation is useful to listen to a transmission between two third-party devices without interfering.

blocking_send (bool)

default: False

When enabled, calling TransportLayer.send() will block until the whole payload has been passed down to the lower layer. In case of failure, a BlockingSendFailure exception will be raised.

Warning

This parameter requires the processing of the transport layer to happen in parallel, therefore TransportLayer.start() must be called prior to send() or a manually generated thread must called process() as fast as possible.

logger_name (str)

default: “isotp”

Sets the name of the logger from the logging module used to log info and debug information

wait_func (callable)

default: “time.sleep”

Defines a waiting function used to create the necessary delay between consecutive frames during a transmission, dictated by the receiver STMin. Expected signature is my_wait_func(delay:float) -> None

Defaults value is the system sleep function, which can have a coarse granularity, depending on the scheduler policy.

Rate Limiter

The isotp.TransportLayer transmission rate limiter is a feature that allows to do some throttling on the output data rate. It works with a simple sliding window and keeps the total amount of bits sent during that time window below the maximum allowed.

The maximum of bits allowed during the moving time window is defined by the product of rate_limit_max_bitrate and rate_limit_window_size. For example, if the target bitrate is 1000b/s and the window size is 0.1sec, then the rate limiter will keep to total amount of bits during a window of 0.1 sec below 100bits.

It is important to understand that this product also defines the maximum burst size that the isotp.TransportLayer object will output, and this is actually the original problem the rate limiter is intended to fix (See issue #61). Consider the case where a big payload of 10000 bytes must be transmitted, after the transmission of the FirstFrame, the receiving party sends a FlowControl message with BlockSize=0 and STMin=0. In that situation, the whole payload can be sent immediately but writing 10000 bytes in a single burst might be too much for the CAN driver to handle and may overflow its internal buffer. In this situation, it is useful to use the rate limiter to reduces the strain on the driver internal buffer.

In the above scenario, having a bitrate of 80000 bps and a window size of 0.1 sec would make the isotp.TransportLayer output a burst of 8000 bits (1000 bytes) every 0.1 seconds.

Warning

The bitrate defined by rate_limit_max_bitrate represent the bitrate of the CAN payload that goes out of the isotp.TransportLayer object only, the CAN layer overhead is excluded. Knowing the a classical CAN message with 11bits ID and a payload of 64 bits usually have 111 bits, the extra 47 bits of overhead will not be considered by the rate limiter. This means that even if the rate limiter is requested to keep a steady 10kbps, depending on the CAN layer configuration, the effective hardware bitrate measured might be much more significant, from 1 to 1.5x more.

Warning

Bitrate is achieved by adding extra wait states which normally translate into OS calls to Sleep(). Because an OS scheduler has a time resolution, bitrate accuracy will be poor if the specified bitrate is very low or if the window size is very small.

Usage

The isotp.TransportLayer object has the following methods

TransportLayer.start()[source]

Starts the IsoTP layer. Starts internal threads that handle the IsoTP communication.

Return type:: None

TransportLayer.stop()[source]

Stops the IsoTP layer. Stops the internal threads that handle the IsoTP communication and reset the layer state.

Return type:: None

TransportLayer.send(data, target_address_type=None, send_timeout=None)

Enqueue an IsoTP frame to be sent over CAN network. When performing a blocking send, this method returns only when the transmission is complete or raise an exception when a failure or a timeout occurs. See blocking_send

Parameters:

data (bytearray | (Generator, int)) – The data to be sent. Can either be a bytearray or a tuple containing a generator and a size. The generator should return integer
target_address_type (int) – Optional parameter that can be Physical (0) for 1-to-1 communication or Functional (1) for 1-to-n. See isotp.TargetAddressType. If not provided, parameter default_target_address_type will be used (default to Physical)
send_timeout (float or None) – Timeout value for blocking send. Unused if blocking_send is False

Raises:

ValueError – Given data is not a bytearray, a tuple (generator,size) or the size is too big
RuntimeError – Transmit queue is full
BlockingSendTimeout – When blocking_send is set to True and the send operation does not complete in the given timeout.
BlockingSendFailure – When blocking_send is set to True and the transmission failed for any reason (e.g. unexpected frame or bad timings), including a timeout. Note that BlockingSendTimeout inherits BlockingSendFailure.

Return type:

None

TransportLayer.recv(block=False, timeout=None)

Dequeue an IsoTP frame from the reception queue if available.

Parameters:

block (bool) – Tells if the read should be blocking or not
timeout (float) – Timeout value used for blocking read only

Returns:

The next available IsoTP frame

Return type:

bytearray or None

TransportLayer.available()

Returns True if an IsoTP frame is awaiting in the reception queue. False otherwise

Return type:: bool

TransportLayer.transmitting()

Returns True if an IsoTP frame is being transmitted. False otherwise

Return type:: bool

TransportLayer.set_address(address)

Sets the layer address. Can be set after initialization if needed. May cause a timeout if called while a transmission is active.

Parameters:: address (Address or AsymmetricAddress) – Address to use
Return type:: None

TransportLayer.stop_sending()[source]

Request the TransportLayer object to stop transmitting, clear the transmit buffer and put back its transmit state machine to idle state.

Return type:: None

TransportLayer.stop_receiving()[source]

Request the TransportLayer object to stop receiving, clear the reception buffer and put back its receive state machine to idle state. If a reception is ongoing, the following messages will be discared and considered like garbage

Return type:: None

Warning

set_address is not thread safe and should be called before start() is called.

Legacy methods (v1.x)

With isotp v2.x, the processing of the transport layer is done from an internal thread. For backward compatibility, the following methods are still accessible to the users, but should not be called from the user thread if start() has been called. It is safe to call them if no call to start() is done.

Calls to non-thread-safe method (reset(), process()) while the internal thread is running will cause an exception to raise.

TransportLayer.reset()[source]

Reset the layer: Empty all buffers, set the internal state machines to Idle

Return type:: None

TransportLayer.process(rx_timeout=0.0, do_rx=True, do_tx=True)[source]

Function to be called periodically, as fast as possible. This function is expected to block only if the given rxfn performs a blocking read.

Parameters:

rx_timeout (float) – Timeout for any read operation
do_rx (bool) – Process reception when True
do_tx (bool) – Process transmission when True

Returns:

Statistics about what have been accomplished during the call

Return type:

ProcessStats

TransportLayer.sleep_time()

Returns a value in seconds that can be passed to time.sleep() when the stack is processed in a different thread.

The value will change according to the internal state machine state, sleeping longer while idle and shorter when active.

Return type:: float

TransportLayer.set_sleep_timing(idle, wait_fc)

Sets values in seconds that can be passed to time.sleep() when the stack is processed in a different thread.

Parameters:

idle (float) – Time when rx state machine is idle
wait_fc (float) – Time when rx state machine is waiting for a flow control message

Return type:

None

Note

The unthreaded transport layer object used in the isotp module v1.x is still accessible under the name isotp.TransportLayerLogic. The isotp.TransportLayer object is an extension of it that can spawn a thread and calls methods that are were to be called by the user.

See Backward Compatibility

Errors

Problem during the transmission/reception are possible. These error are reported to the user by calling a error handler and passing an error object containing the details of the error. An error handler should be a callable function that expects an isotp.IsoTpError as first parameter.

Warning

The error handler will be called from the internal thread, therefore, any interaction with the application should use a thread safe mechanism

my_error_handler(error)

Parameters:: error (isotp.IsoTpError) – The error

All errors inherit isotp.IsoTpError which itself inherits Exception

class isotp.FlowControlTimeoutError(*args, **kwargs)[source]

Happens when the senders fails to sends a Flow Control message in time. Refer to TransportLayer parameter rx_flowcontrol_timeout

Parameters:

args (Any) –
kwargs (Any) –

class isotp.ConsecutiveFrameTimeoutError(*args, **kwargs)[source]

Happens when the senders fails to sends a Consecutive Frame message in time. Refer to TransportLayer parameter rx_consecutive_frame_timeout

Parameters:

args (Any) –
kwargs (Any) –

class isotp.InvalidCanDataError(*args, **kwargs)[source]

Happens when a CAN message that cannot be decoded as valid First Frame, Consecutive Frame, Single Frame or Flow Control PDU is received.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.UnexpectedFlowControlError(*args, **kwargs)[source]

Happens when a Flow Control message is received and was not expected

Parameters:

args (Any) –
kwargs (Any) –

class isotp.UnexpectedConsecutiveFrameError(*args, **kwargs)[source]

Happens when a Consecutive Frame message is received and was not expected

Parameters:

args (Any) –
kwargs (Any) –

class isotp.ReceptionInterruptedWithSingleFrameError(*args, **kwargs)[source]

Happens when the reception of a multi packet message reception is interrupted with a new Single Frame PDU.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.ReceptionInterruptedWithFirstFrameError(*args, **kwargs)[source]

Happens when the reception of a multi packet message reception is interrupted with a new First Frame PDU.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.WrongSequenceNumberError(*args, **kwargs)[source]

Happens when a consecutive frame is received with a wrong sequence number.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.UnsupportedWaitFrameError(*args, **kwargs)[source]

Happens when a Flow Control PDU with FlowStatus=Wait is received and wftmax is set to 0

Parameters:

args (Any) –
kwargs (Any) –

class isotp.MaximumWaitFrameReachedError(*args, **kwargs)[source]

Happens when too much Flow Control PDU with FlowStatus=Wait is received. Refer to wftmax

Parameters:

args (Any) –
kwargs (Any) –

class isotp.FrameTooLongError(*args, **kwargs)[source]

Happens when a FirstFrame with a length (FF_DL) longer than max_frame_size is received.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.ChangingInvalidRXDLError(*args, **kwargs)[source]

Happens when a ConsecutiveFrame is received with a length smaller than RX_DL (size of first frame) without being the last message of the IsoTP frame.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.MissingEscapeSequenceError(*args, **kwargs)[source]

Happens when a SingleFrame with length (CAN_DL) greater than 8 bytes is received and the length of the payload (SF_DL) is encoded in the first byte, which is forbidden by ISO-15765-2

Parameters:

args (Any) –
kwargs (Any) –

class isotp.InvalidCanFdFirstFrameRXDL(*args, **kwargs)[source]

Happens when a FirstFrame is received with missing data; In other words when CAN_DL is smaller than the deduced RX_DL. The sender did not optimized the capacity usage of the CAN message.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.OverflowError(*args, **kwargs)[source]

Happens when the TransportLayer receive a FlowControl PDU with a FlowStatus=Overflow (2). In this event, the transmission is stopped.

Parameters:

args (Any) –
kwargs (Any) –

class isotp.BadGeneratorError(*args, **kwargs)[source]

Happens when the user tries to send data using a generator and provides a size that does not match the amount of data in the generator

Parameters:

args (Any) –
kwargs (Any) –

Exceptions

Some exception can be raised in special cases. These are never sent to the error handler

class isotp.BlockingSendFailure(*args, **kwargs)[source]

Happens when a blocking send fails to complete

Parameters:

args (Any) –
kwargs (Any) –

class isotp.BlockingSendTimeout(*args, **kwargs)[source]

Happens when a blocking send fails to complete because the user timeout is expired. Inherits BlockingSendFailure

Parameters:

args (Any) –
kwargs (Any) –

Note

BlockingSendTimeout inherits BlockingSendTimeout. Catching a BlockingSendFailure will also catch timeouts

About timings

Timing management is a complex subject and has been the source of a lot of discussions in Github issue tracker. v1.x had many timing-related flaws that were addressed in v2.x. Still, reaching good timing with the pure python TransportLayer object is not an easy task, mainly because of the 2 following facts

Timing depends on the OS scheduler

The transport layer is not tightly coupled with the underlying layers (rxfn is user provided)

The fact #2 is useful for compatibility, allowing to couple the isotp layer with any kind of link layer. Unfortunately, it has the drawback of preventing cross-layer optimizations. For that reason, this module employs a 3 thread strategy and rely on the python Queue object for synchronization. The python Queue module employ OS primitives for synchronization, such as condition variables to pass control between threads, which are as performant as they can be.

See the following figure

The relay thread reads the user provided rxfn in a loop and fills the relay queue when that callback returns a value different from None
The worker thread interact with the user by doing non-blocking reads to the Rx Queue
The worker thread does blocking reads to the relay queue and gets woken up right away by Python when a message arrives
When the user calls send(), a None is injected in the relay queue, forcing the worker thread to wake up and process the user provided payload right away.

Using the approach described above, a message can be read from the link-layer and processed after 2 context switches, which are achievable in about 20us each on both Windows and Linux. This 40us latency is far better than the latency caused by calls to time.sleep() required with v1.x. Considering that a CAN bus running at 500kbps has a message duration of about 230us, the latency is in the acceptable range.

Finally, the delay between consecutive frames is dictated by a user-definable function passed with the wait_func parameter. By default, the wait function is the system sleep function. On machines where the sleep function has a coarse granularity and a high resolution timer is available, it is possible to pass a busy-wait function to this parameter.

Backward compatibility

For backward compatibility with v1.x, the isotp.TransportLayer is an extension of the v1.x transport layer, which has been renamed to TransportLayerLogic. In v1.x, the user had to handle timing and repeatedly call the process() method. To avoid breaking applications using that approach, the isotp.TransportLayer inherits TransportLayerLogic, therefore the old interface is still accessible under the same name. Inheritance is used instead of composition for that purpose.

When calling start() and stop(), which have been added in the 2.x extension, it is assumed that the user will uses the TransportLayer as documented by the v2.x documentation, otherwise race conditions could occur. Put simply, process() should never be called after start() has been called, otherwise, an exception will be raised.