DRAFT: ZMQ messages from HALUI to GUIs

[c-morley]

This is the on going work started here:
#3528
This generalizes the idea for messages.

There are three proof of concept sims:
axis_halui_test.ini
gmoccapy_halui_test.ini
qtdragon_halui_test.ini

They will load the respective gui and a separate simulated control panel.
You can notice the jograte slider on the control panel changes the jog rate on the gui.
I would love it if someone could test these rough changes.

[zz912]

Would it be possible to send a string containing Gcode via ZMQ?

Then it would be possible to make a really useful interface for LCNC.
Then it would be possible to control LCNC meaningfully via ZMQ.
Whether it is HALUI, Probing, scientific applications that need a manipulator, a higher-level robot system, Camera object recognition, Artificial intelligence ...

[c-morley]

Yes you could send gcode via ZMQ. I don't know what the limits of the amount of data you could send at once.
I have a branch for ncam integration that uses ZMQ in Qtdragon and Gmoccapy to load the Ncam program after each update. In this case it sends a function name to call in the GUI and the arguments of the function. It works well. Not quite what you mean bit similar.

A version of versa probe is actually already in linuxcnc's repo:
https://github.com/LinuxCNC/linuxcnc/tree/master/lib/python/gladevcp/builtin-panels/versa-probe
it hasn't had any testing in a long while, so I don't know if it works properly or not.

[rmu75]

Seems I somehow missed that discussion. Just wanted to say I did a fairly complete POC to add a 0mq messaging layer parallel to NML, so you could run linuxcnc without any NML. see https://github.com/rmu75/linuxcnc/tree/rs/zmq-experiments.

[zz912]

rmu75:
What does POC mean?

c-morley:

Not quite what you mean bit similar.

I dont agree. I think that halui.mdi-command, versa-probe, ncam, ...... should have same interface for GUI.
This interface should resolve issues related to mode switching and race-conditions with other commands.
I would limit the string to 128 chars, for example, so that ZMQ doesn't send long messages.
If you need to run something bigger, you'll use "o call".

[rmu75]

rmu75: What does POC mean?

Proof of Concept.

machinekit did add a bridge between NML and something 0mq/protobuf based, probably because they didn't want to touch the NML layer.

I wanted to explore if replacing NML would be feasible.

[c-morley]

Seems I somehow missed that discussion. Just wanted to say I did a fairly complete POC to add a 0mq messaging layer parallel to NML, so you could run linuxcnc without any NML. see https://github.com/rmu75/linuxcnc/tree/rs/zmq-experiments.

How well did it work?

[c-morley]

Not quite what you mean bit similar.

I dont agree. I think that halui.mdi-command, versa-probe, ncam, ...... should have same interface for GUI. This interface should resolve issues related to mode switching and race-conditions with other commands. I would limit the string to 128 chars, for example, so that ZMQ doesn't send long messages. If you need to run something bigger, you'll use "o call".

The interface can not resolve mode switching problems by it's self. Only the GUI programmer can do that.
What the interface changes is that it 'asks' the GUI to do something rather then doing something on it's own and hoping the GUI notices and does the right thing.

It certainly can ask the GUI to run some gcode, if that is what you want.

[zz912]

Not quite what you mean bit similar.

I dont agree. I think that halui.mdi-command, versa-probe, ncam, ...... should have same interface for GUI. This interface should resolve issues related to mode switching and race-conditions with other commands. I would limit the string to 128 chars, for example, so that ZMQ doesn't send long messages. If you need to run something bigger, you'll use "o call".

The interface can not resolve mode switching problems by it's self. Only the GUI programmer can do that. What the interface changes is that it 'asks' the GUI to do something rather then doing something on it's own and hoping the GUI notices and does the right thing.

I thought "interface" + "modification GUI for interface" should be same for halui.mdi-command, versa-probe, ncam, ......

I thought "modification GUI for interface" should resolve issues related to mode switching and race-conditions with other commands.

It certainly can ask the GUI to run some gcode, if that is what you want.

I want:

• ask the GUI to run some gcode
• send string of gcode to GUI

But I'm not a Gmoccapy developer, so it doesn't matter to me.

[rmu75]

Seems I somehow missed that discussion. Just wanted to say I did a fairly complete POC to add a 0mq messaging layer parallel to NML, so you could run linuxcnc without any NML. see https://github.com/rmu75/linuxcnc/tree/rs/zmq-experiments.

How well did it work?

basically it does work. I tested with a minimal debug-tool https://github.com/rmu75/cockpit/tree/rs/zmq in parallel to axis & co. Overhead is slightly higher than with NML, because zeromq doesn't have a shared memory transport, so everything is serialized/desrialized into a pipe, but nothing really noticable.

The tedious part, extraction of all messages into IDL files and the writing the boilerplate code turned out to be relatively quick and easy. one central point needed changing of a pointer to std::unique_ptr because messages received via zeromq are dynamically allocated and need to be disposed of whereas NML messages always get copied to some static buffer. That change already is in master. 6d65386

The zeromq messaging layer would allow for some improved semantics of e.g. the error / warning channel, at the moment, only one consumer is allowed, if you connect more than one it is defacto undeterministic which one receives a message.

[andypugh]

A few notes:

I don't know if the probe-screen in the LinuxCNC repository has been updated for Python3 / Gtk3. I did that task in a fork of a fork of psng: https://github.com/andypugh/probe-screen-ng

There was a plan some time ago to replace NML with 0MQ/ZMQ. I recall a phone call from a meetup in Wichita to the original author of NML at NIST. He was astonishd that we were still using it and thought that switching to zmq was a good idea.

But that got all tangled up in the Machinekit split, and I don't think that MK ever got round to doing it.

[Sigma1912]

@rmu75
How much work do you think your proof of concept would need to be able to be merged?

[rmu75]

In principle it could be merged as is, it can be run parallel to NML. Modulo bugs and things I missed. Some infrastructure / configurability and tests have to be done. Maybe alternatives to 0mq resp. flatbuffers should be evaluated before merging: (nanomsg, protocol buffers, capt'n'proto) -- flatbuffers seemed to have lower overhead compared to protobuf.

[rmu75]

It seems you are manually serializing the messages into JSON. Is there a particular reason you don't use something like protocol buffers, capt'n'proto or flatbuffers? Compare with the current NML situation, at some point a tool did exit that generated the boilerplate serialization/deserialization code, but that tool seems to have disappeared and changes to messages are tedious and error-prone.

IMO it would be cleaner to use some tool that takes some kind of interface description (IDL-file) and generated code for serialization/deserialization. That would also make it easier to talk to other languages.

[c-morley]

Thanks for looking at the code and commenting. I used JSON as that was the example that I found and it was simple. At the moment I'm trying to flesh out the idea/process of adding this utility to HALUI and see if it's as useful as I think it is.
What you are suggesting is beyond my programming experience.
Can you show me a reference source or a code example to help?

[rmu75]

I will rebase and fix my 0mq nml/rcs replacement branch in the next few weeks, then there should be a runnable example.

I used flatbuffers to build the wire format https://flatbuffers.dev/, other candidates would be google protocol buffers and capt'n'proto. Protocol buffers seem to have slightly more overhead but are way more popular.

https://github.com/rmu75/linuxcnc/blob/rs/zmq-experiments/src/emc/task/emctaskmain.cc#L239 shows how to put linuxcnc error message into a message and send it out to zmq, https://github.com/rmu75/linuxcnc/blob/rs/zmq-experiments/src/emc/flatbuf/emc_error.fbs is the corresponding interface definition.

this https://github.com/rmu75/cockpit/blob/rs/zmq/src/zmqcom.cpp shows how the equivalent to shcom.cc could look like.

I don't yet have any python-code examples, will have to refer to this https://flatbuffers.dev/tutorial/#python.

It would be really nice if we could get rid of direct HAL usage of the diverse GUIs and use messages / remote procedure calls instead. Would make building some sort of remote control / remote GUI / remote DRO / android pendant app etc... much easier.

[c-morley]

I don't yet have any python-code examples, will have to refer to this https://flatbuffers.dev/tutorial/#python.

It would be really nice if we could get rid of direct HAL usage of the diverse GUIs and use messages / remote procedure calls instead. Would make building some sort of remote control / remote GUI / remote DRO / android pendant app etc... much easier.

Ok thanks I'll look at that. I could library out the messages, so any technology could be used.
Yes getting to a message based GUI interaction seems like the right idea, though I'm not sure we would every get to no HAL pins. Really if NML was more digestible/documented I would have tried that way.

[zz912]

Hi Chris,

I wanted to make an improvement for LinuxCNC. Unfortunately, I was limited by the current technical capabilities of ChatGPT. I worked with ChatGPT for two days to architecture my improvement , and we programmed it together for another two days, and it ended up taking about half an hour for each of its responses, and ChatGPT got confused. However, we ended up a little short of our goal. When I have time, I will study everything ChatGPT created and finish it manually. I don't have time for my improvement at the moment. Why am I writing this to you? Thanks to ChatGPT, I have understood more deeply how LCNC works. I have become more familiar with the task module.

Based on newly acquired information, I conclude that all of our images above were wrong.

• GUI should never interact with the motion module directly (perhaps you meant a task by the word motion controller)
• The GUI should never interact with the halui module directly.

I made a picture based on the official architecture:
https://linuxcnc.org/docs/devel/html/code/code-notes.html#_architecture_overview

(I think this picture has a lot of mistakes, but it's enough for start.)
The heart of LCNC is TASK. It makes sense and I think we should follow it.

Merely replacing NML technology with ZMQ technology is not interesting to me. However, integrating all the requirements that arose during the development of LCNC into ZMQ technology would be interesting to me. I could help with creating the architecture (drawing blocks), writing and updating documentation, and testing.

Above I asked the question why HAL is not used everywhere. Now I will answer this question:

• HAL should be the interface between core LCNC and interface LCNC
  -- core LCNC is the part of LCNC that developers work on and is universal for all CNC machines
  -- interface LCNC is the part of LCNC that integrators work on and is intended for a specific machine

• HAL is often used in GUI, this is caused either by a poor understanding of NML, or by the non-existent prohibition of HAL in GUI, or by insufficient capabilities of NML
  I think that if ZMQ was to be integrated into LCNC, it should be modified so that it is not necessary to use HAL in GUI.

I think that NML was done well, but at the time when it was created, current GUIs did not exist. At the time when current GUIs were created, NML was not adapted.
Now we can adapt ZMQ to GUIs.

I used to look for a problem with switching modes everywhere. Today I am convinced that the problem is in the TASK module.
Nowadays, when we want to execute G-code, we can only use self.command.mdi("string"). This command requires switching modes. Unfortunately, manual switching modes outside of TASK causes problems such as race conditions. I think there should be a command self.command.gcode("string"), which would coexist with self.command.mdi("string"), but would not require switching modes. TASK would arrange for the execution of G-code and would not bother the developers of the GUI and other modules by switching modes. Maybe it would be necessary to make a signal about the execution of the gcode. However, that signal should come from one place. It should come from TASK.
I think you already wrote that you managed to send g-code via ZMQ. So it may not be the command self.command.gcode("string"), but it will look completely different.

I wanted to ask you, what is your priority now?
Understanding ZMQ technology or proper integration, or both?

I think proper ZMQ integration should be addressed in LCNC 2.11. Because it will affect everything.

I think version 2.11 should do:

• clean up files in folders *.glade and other files in /lib/python/gladevcp/ #3609 probe_icons - duplicate files #3608
• ZMQ integration
• remove unmaintained GUIs

[c-morley]

Well that's quite a lot to digest.
This pr does not intend to replace NML with ZMQ.
I am not really capable of the work needed to do that anyways.
I think it would probably be a good idea to get away from NML.

The problem I am trying to fix with this pr is with the fact that every UI has a slightly different way to connect HAL pins for common required behaviour. HALUI has turned into a fairly standard way to connect physical operator panel to linuxcnc. Unfortunately it also does things it's own way that does not integrate well with the other GUIs.
So the idea here is that HALUI handles the physical button inputs and sends a message to the GUI, so that the GUI can do what ever it was programmed to do.
This then puts things like task switches mostly in the GUI code, so it knows whats going on.

IMHO mode switching between manual, mdi and auto is an old concept that could mostly be done away with. We are constantly working around the problem by switching modes then immediately switching back which is silly and error prone.
But that is another big subject that I doubt anyone will take on.

[zz912]

Well that's quite a lot to digest.

On the other hand, we don't have a deadline for anything.

This pr does not intend to replace NML with ZMQ. I am not really capable of the work needed to do that anyways. I think it would probably be a good idea to get away from NML.

I agree that this PR is not intended to be a general NML/ZMQ exchange. However, we can continue the debate elsewhere.

The problem I am trying to fix with this pr is with the fact that every UI has a slightly different way to connect HAL pins for common required behaviour. HALUI has turned into a fairly standard way to connect physical operator panel to linuxcnc. Unfortunately it also does things it's own way that does not integrate well with the other GUIs. So the idea here is that HALUI handles the physical button inputs and sends a message to the GUI, so that the GUI can do what ever it was programmed to do. This then puts things like task switches mostly in the GUI code, so it knows whats going on.

Do you really want to create a communication triangle? HALUI <=> GUI <=> TASK <=> HALUI
Another problem will be that this communication triangle will be formed by three technologies: NML, ZMQ, HAL.
I understand that you don't want to take big actions, but your bridge "ZMQ messages from HALUI to GUIs" is just another bypass of the problem. We already have bypass number 1 using HAL, and you will create bypass number 2.

IMHO "ZMQ messages from HALUI to GUIs" is not good idea. I apologize if my opinions are rude. My English skills don't allow me to write this more politely. I know this PR has taken you a lot of work.

IMHO mode switching between manual, mdi and auto is an old concept that could mostly be done away with. We are constantly working around the problem by switching modes then immediately switching back which is silly and error prone. But that is another big subject that I doubt anyone will take on.

On the other hand, more people are trying to solve this problem by bypassing it. You created Bridge, I created HAL pins "is_running", @Sigma1912 created native HAL pins in Gmoccapy. We all made the mistake of not trying to understand/modify TASK and so our solutions are imperfect.

[c-morley]

HAL and NML are for two different uses and are not really interchangeable. NML and ZMQ are similar and could be interchanged.

So yes if NML was easier to use, I might have used it instead of ZMQ for this pr.

The reasons for having the GUI communicate to HALUI:

1. It's the defacto standard way to connect HAL to a control panel.
2. it does not preclude using sinething else, where as if you put it in task you are quite stuck with what we decide.
3. it's small/simple enough I can see it through
4. it should not break configuration that use the current way, at least for now.
5. there is information that task/motion don't know, such as jog rate. Currently if you use HALUI and a GUI then there are two jog rates.