Dear friends!
Thanks to everyone who showed interest in my project Mooha!
After the publication of the first article with the announcement I received a lot of feedback, critical and useful comments, as well as interesting thoughts about the use of the system. The work on the project is not finished yet, but the development is in the final stage. I will let you know when the first version is ready for beta testing. I hope that this will happen before the end of the year, but it is difficult to name the exact date, because I only work on this project in my spare time.
Thank you for your willingness to participate in the testing! I hope that with your support and your suggestions it will be possible to create a really useful and easy-to-use tool.
Friday, 22 November 2013
Monday, 11 November 2013
Mooha – Node Interface for PHP
I often had to deal with node interfaces in various programs. Starting from music modular software and finishing with the packages for creating 3D graphics.
The idea of graphical control in a program has always seemed to me a very elegant, and in some cases, the only good solution. Later, when besides music and video, I became interested in programming (mostly PHP), I wanted to try, even as an experiment, to create a graphic interface for solving those little creative challenges that I faced in my work.
My acquaintance with the node interface started with the program Plogue Bidule. This application belongs to the family of modular programs to control the sound in real time. At the time, this program helped me to handle rather nontrivial problems related to sound processing and stage equipment sync for the live performances of several of my musical projects.
The interface of the program Plogue Bidule looks as follows:
At the heart of it lies creation of logical chains. To my mind, this approach fits naturally into the work with sound, although for some it may seem complicated at first glance. The sound in this program is a continuous stream which emerges from the source (input channel of a sound card, audio file or signal generator), and after certain processing often goes to the output of the sound card or to the file recording.
The main interface control constructions are nodes (or modules, blocks, components, etc.), which are in fact functions. By analogy with the music equipment — these are devices interconnected with wires.
Plogue Bidule includes many interesting features which are not restricted only to work with the sound, but for this very article this description will suffice.
I was lucky to face node interface for the first time in a music program, since the flow of sound, in my opinion, is an excellent analogy for the flow of procedural program execution.
Node editors are widely used in graphics packages for the creation of 3D graphics. In such programs, the "black boxes" describe complex structures and functions, and the interface allows you to preserve the freedom and flexibility to build the logic of realization. The most interesting, in my view, is the implementation of node system in the program Houdini.
So what about web programming? The search of similar systems gave no result. I thought that perhaps there is simply no use in this technology and that maybe this idea has come to me more because of my incompetence and naivety. But eagerness to try to create such a tool was stronger than my doubts.
That's how the project Mooha was born.
How is it possible to combine these three technologies in one node system to get a user-friendly graphical tool? The programs for the creation of 3D graphics in node editors often use separation of the contexts that are usually poorly compatible with each other. But in our case, the data structures are not so complex and diverse to produce any separate editors. So I decided to create a common space, dividing the nodes into the following technology categories:
Let's consider all three categories of nodes.
When combining these nodes together, we get a simple HTML document structure. By default, HTML / XML node has one input and one output. The input (IN) is something that is put into a tag, the output (OUT) is a resulting string. Thus, the last "HTML" node output gives us the line:
A service node "Merge" is used to influence the sorting of incoming connections. Sorting is available in the properties of the node itself. In the last "HTML" node two incoming connections are concatenated in the order in which there was a connection. That is, if you first connect the "HEAD", and then "BODY", that will be the order in which they will appear in the final string.
Tag attributes can be specified in the properties of the nodes, but optionally they can be set as connectors for incoming flow. For example, adding a node "my div" the connectors for the id and class attributes, we get the following form:
The variables with string values "my-id" and "my-class" become our attributes. The variable "my text" becomes the content of the first div tag. The variables are PHP nodes, which will be described below.
We can also output templates as connectors to replace the text inside the tag. For example, if we have the following tag with assigned content:
and if the pattern %user% is put out as a connector, we can replace this template with a variable, as follows:
In the first case, the text inside the node is assigned in its properties, the connector for the data is turned off and only the connector for replacing a template appears. In the second case, the text with the template also comes from the variable. In both cases, in the output we get:
Of course, there is not much sense in the construction of HTML code that way. The editor has a possibility to use not only the nodes with individual tags, but also with whole pieces of code, and even connect them to external files.
For example, we have a file template.tpl, which contains two templates for replacement: %title% and %content%. The contents of the file is the following:
To attach a file and replace the template with variables it's enough to just build such a scheme:
As a result, we get:
Since we have already gone beyond the description of HTML nodes, let's move on to the next type — to PHP nodes.
The first three nodes, "1", "2" and "5" are the variables, the values of which correspond to the names.
As a result we get the following script:
And in the output we get 2.
I tried to implement the necessary logic constructions that are commonly used in node systems. For example, here's what a comparison of two variables (or the results of two streams) looks like:
The result will be true (1!= 2).
Here's how you can make a choice of flow, based on the conditions:
Here values are assigned to each red connector in the node properties. These values are compared with the value received by orange connector. If the values match, the Switch transmits the corresponding flow. In this example red plugs have values 1, 2 and 3. That means that the result of this construction is "it was 2."
Connections of all the chains transfer the final values of all previous calculations of the flow. Thus in the following example we will also get the result "it was 2", but the resulting string will consist of several actions:
Something like the following code is generated:
Node Copy should be given special attention, as it clones the template from the second connection according to the values of the first one. This is useful, for example, in the case when we want to clone HTML code and replace the templates with the values from the array. In the simplest case of one-dimensional array the scheme looks as follows:
For clarity I named the nodes with the values of the variables. We link JSON array to the first connector (the one for the values) and HTML code with replacement template to the second one. All the nodes (or node chains) linked to the template connector will be copied as many times as there are elements in the array that is linked to the first connector. Each node will have a special setting on the left of the connection to specify which kind of data should be replaced by an element of the array. In this case, replacement of the template %number% is set in the node with paragraph. That means, as a result of this scheme we will get the following code:
Here is a more complex example with an array of objects and multiple nodes:
In the variable "JSON" there is a following array of objects:
The object "title" of the array is specified as a source for copying for the template %title% in the node with <h1> tag. The object "content" of the array is specified as a source for copying for the template %content% in the node with the tag <p>. As a result, the following code is generated:
On default Copy node has the type foreach and works in the way that I described in the two examples above. But we can also choose the type for, after which it will be possible to configure initial value of the counter, final value, step and type of the increment (ascending or descending).
Nodes of the Queries to MySQL Database.
The base node of query is a table or rather the operator SELECT, which is executed by default when connecting to any field. By connecting to the last output, marked with "*", we execute the query:
We get the result as an array of JSON objects. This connection will "catch" the next data:
To get an array of only the authors' names, the connection will look the following way:
On the output, we will get a usual one-dimensional array:
Currently in the editor there are several official SQL nodes, with the help of which you can carry out simple queries. For example:
In the first case (on the left), we execute the following query:
And we will get an array with one object:
In the second case (on the right) the query will be the following:
Here is the query with multiple conditions and an opportunity of sorting:
AND node acts here not only as a logical operator in WHERE condition, but also as a connection of commands in one query (in this case, it is the addition to the query of the command ORDER BY)
Queries INSERT, UPDATE, and DELETE are sent with the connections to the same chart-nodes, but only with appropriate settings.
Simple INSERT looks like this:
UPDATE:
DELETE:
In the process of construction, the queries using PDO are generated.
With the help of node connections almost any type of queries can be built, including organizing JOIN unions. However, I haven't found the ideal graphic solution for all the complex queries, so I left the opportunity to write the text of a SQL statement in a special node. Looking ahead, I should say that the work with database can be built in the system in different ways, not only with the help of MySQL nodes.
Finally, the example of combining all three technologies (MySQL, PHP and HTML) in the same chain may look as follows:
Here we have a ready library.tpl template for showing the books from home library. The content is something like that:
We have two templates for replacement: %author-list% and %book-list%. In the first one we will have to put option tags, corresponding to all the authors. In the second one we need to put all the books, namely to clone "book block", which contains the HTML code for the output of a book. We need the first list to filter the books by author (for this we will use JQuery function in the template library.tpl).
For the first template %author-list% we add the node Copy ("Author Copy"), which clones the tag "option" according to the data from the table "authors". At the same time it places the name of the author into the tag, and assigns the value of id to "value" attribute. In the output of "Author Copy" node we get the following code:
"book block" node contains the following code:
The templates in this section are replaced with book data from joined records of two tables "books" and "authors". Table "authors" is here to display the name of the author in the book . The block itself is cloned as many times as we have records in the table "books".
That's what we get as a result:
In this article I have tried to describe the basic idea of the work of the system. In the project there are lots of other possibilities implemented. Some of them are typical for node interfaces. Others I had to invent on my own, taking into account the particular characteristics of the editor. Here is a noninclusive list of the currently available tools:
The result of the work of the editor is a set of generated PHP files. Each scene is a separate script that is saved in a special folder. Once the script has been created, it can be executed without the help of the editor. In addition to that, in the system settings a start-up script can be selected, which includes the main logic of the whole project, based on the data (query string or sent parameters).
The purpose of writing this article is to understand whether such a system could be of interest to developers. I was enthusiastically engaged in this project for too long and from the moment I started working on it my attitude to it has changed. Therefore, I would be most grateful for any feedback and comments, for any questions and suggestions, as well as for participating in testing.
At the moment the project is almost complete and now it is in the process of debugging. Very soon I will be ready to share the beta version. In order to stay informed about the process of development and learn about the release of the project you just need to leave your email address in the appropriate form on the site Mooha.net.
In the project the following tools were used:
The server component was tested in the following conditions:
The idea of graphical control in a program has always seemed to me a very elegant, and in some cases, the only good solution. Later, when besides music and video, I became interested in programming (mostly PHP), I wanted to try, even as an experiment, to create a graphic interface for solving those little creative challenges that I faced in my work.
My acquaintance with the node interface started with the program Plogue Bidule. This application belongs to the family of modular programs to control the sound in real time. At the time, this program helped me to handle rather nontrivial problems related to sound processing and stage equipment sync for the live performances of several of my musical projects.
The interface of the program Plogue Bidule looks as follows:
The interface of the program Plogue Bidule |
At the heart of it lies creation of logical chains. To my mind, this approach fits naturally into the work with sound, although for some it may seem complicated at first glance. The sound in this program is a continuous stream which emerges from the source (input channel of a sound card, audio file or signal generator), and after certain processing often goes to the output of the sound card or to the file recording.
The main interface control constructions are nodes (or modules, blocks, components, etc.), which are in fact functions. By analogy with the music equipment — these are devices interconnected with wires.
Guitar pedals and effects |
Plogue Bidule includes many interesting features which are not restricted only to work with the sound, but for this very article this description will suffice.
I was lucky to face node interface for the first time in a music program, since the flow of sound, in my opinion, is an excellent analogy for the flow of procedural program execution.
Node editors are widely used in graphics packages for the creation of 3D graphics. In such programs, the "black boxes" describe complex structures and functions, and the interface allows you to preserve the freedom and flexibility to build the logic of realization. The most interesting, in my view, is the implementation of node system in the program Houdini.
The interface of the Houdini |
That's how the project Mooha was born.
PHP, MySQL, HTML
How is it possible to combine these three technologies in one node system to get a user-friendly graphical tool? The programs for the creation of 3D graphics in node editors often use separation of the contexts that are usually poorly compatible with each other. But in our case, the data structures are not so complex and diverse to produce any separate editors. So I decided to create a common space, dividing the nodes into the following technology categories:
- HTML/XML nodes (for the output of HTML / XML tags)
- PHP nodes (logic management)
- Nodes of the queries to MySQL database
Let's consider all three categories of nodes.
HTML/XML Nodes
When combining these nodes together, we get a simple HTML document structure. By default, HTML / XML node has one input and one output. The input (IN) is something that is put into a tag, the output (OUT) is a resulting string. Thus, the last "HTML" node output gives us the line:
<html> <head> <meta /> <script /> </head> <body> <div> <div /> <div /> </div> </body> </html>
A service node "Merge" is used to influence the sorting of incoming connections. Sorting is available in the properties of the node itself. In the last "HTML" node two incoming connections are concatenated in the order in which there was a connection. That is, if you first connect the "HEAD", and then "BODY", that will be the order in which they will appear in the final string.
Tag attributes can be specified in the properties of the nodes, but optionally they can be set as connectors for incoming flow. For example, adding a node "my div" the connectors for the id and class attributes, we get the following form:
<div id="my-id" class="my-class"> <div>my text</div> </div>
The variables with string values "my-id" and "my-class" become our attributes. The variable "my text" becomes the content of the first div tag. The variables are PHP nodes, which will be described below.
We can also output templates as connectors to replace the text inside the tag. For example, if we have the following tag with assigned content:
<h1>Hello, %user%</ h1>
and if the pattern %user% is put out as a connector, we can replace this template with a variable, as follows:
In the first case, the text inside the node is assigned in its properties, the connector for the data is turned off and only the connector for replacing a template appears. In the second case, the text with the template also comes from the variable. In both cases, in the output we get:
<h1>Hello, Mooha</ h1>
Of course, there is not much sense in the construction of HTML code that way. The editor has a possibility to use not only the nodes with individual tags, but also with whole pieces of code, and even connect them to external files.
For example, we have a file template.tpl, which contains two templates for replacement: %title% and %content%. The contents of the file is the following:
<html> <head> </head> <body> <h1>%title%</h1> <div>%content%</div> </body> </html>
To attach a file and replace the template with variables it's enough to just build such a scheme:
As a result, we get:
<html> <head> </head> <body> <h1>My article title</h1> <div>My article content</div> </body> </html>
Since we have already gone beyond the description of HTML nodes, let's move on to the next type — to PHP nodes.
PHP Nodes
This is the simplest example of arithmetic calculation and standard function call abs() with the use of PHP nodes. Node names are random and they may be changed in the properties of every node. For convenience, I renamed them so that you can see the values of variables and names of arithmetic operations.The first three nodes, "1", "2" and "5" are the variables, the values of which correspond to the names.
As a result we get the following script:
echo abs((1+2)-5);
And in the output we get 2.
I tried to implement the necessary logic constructions that are commonly used in node systems. For example, here's what a comparison of two variables (or the results of two streams) looks like:
The result will be true (1!= 2).
Here's how you can make a choice of flow, based on the conditions:
Here values are assigned to each red connector in the node properties. These values are compared with the value received by orange connector. If the values match, the Switch transmits the corresponding flow. In this example red plugs have values 1, 2 and 3. That means that the result of this construction is "it was 2."
Connections of all the chains transfer the final values of all previous calculations of the flow. Thus in the following example we will also get the result "it was 2", but the resulting string will consist of several actions:
Something like the following code is generated:
$data = 2; switch ($ data) { case "1" // set in the first connector of the comparison value $result = "it was 1"; break; case "2" :// connector specified in the second comparison value $result = str_replace ("foo", "2", "in was foo"); break; case "3": // set the value of the third connector to compare $result = "it was 3"; break; default: $result = false; } var_dump ($result);
Node Copy should be given special attention, as it clones the template from the second connection according to the values of the first one. This is useful, for example, in the case when we want to clone HTML code and replace the templates with the values from the array. In the simplest case of one-dimensional array the scheme looks as follows:
For clarity I named the nodes with the values of the variables. We link JSON array to the first connector (the one for the values) and HTML code with replacement template to the second one. All the nodes (or node chains) linked to the template connector will be copied as many times as there are elements in the array that is linked to the first connector. Each node will have a special setting on the left of the connection to specify which kind of data should be replaced by an element of the array. In this case, replacement of the template %number% is set in the node with paragraph. That means, as a result of this scheme we will get the following code:
<p>first paragraph</p> <p>second paragraph</p> <p>third paragraph</p>
Here is a more complex example with an array of objects and multiple nodes:
In the variable "JSON" there is a following array of objects:
[ {"Title": "first title", "content": "first content"}, {"Title": "second title", "content": "second content"} ]
The object "title" of the array is specified as a source for copying for the template %title% in the node with <h1> tag. The object "content" of the array is specified as a source for copying for the template %content% in the node with the tag <p>. As a result, the following code is generated:
<div> <h1>first title</h1> <p>first content</p> </div> <div> <h1>second title</h1> <p>second content</p> </div>
On default Copy node has the type foreach and works in the way that I described in the two examples above. But we can also choose the type for, after which it will be possible to configure initial value of the counter, final value, step and type of the increment (ascending or descending).
Nodes of the Queries to MySQL Database.
The base node of query is a table or rather the operator SELECT, which is executed by default when connecting to any field. By connecting to the last output, marked with "*", we execute the query:
SELECT * FROM `authors`
We get the result as an array of JSON objects. This connection will "catch" the next data:
[ {"Id": "1", "Name": "Iain Menzies Banks", "Birthday": "1954-02-16"}, {"Id": "2", "Name": "Charles Michael Palahniuk", "Birthday": "1962-02-21"} ]
To get an array of only the authors' names, the connection will look the following way:
On the output, we will get a usual one-dimensional array:
["Iain Menzies Banks", "Charles Michael Palahniuk"]
Currently in the editor there are several official SQL nodes, with the help of which you can carry out simple queries. For example:
In the first case (on the left), we execute the following query:
SELECT * FROM `authors` WHERE id = 1
And we will get an array with one object:
[ {"Id": "1", "Name": "Iain Menzies Banks", "Birthday": "1954-02-16"} ]
In the second case (on the right) the query will be the following:
SELECT * FROM `authors` WHERE Name LIKE '% Banks%'
Here is the query with multiple conditions and an opportunity of sorting:
SELECT * FROM `authors` WHERE (`Birthday` <'1990-01-01 'AND `Birthday`> '1920-01-01') ORDER BY `Name` ASC
AND node acts here not only as a logical operator in WHERE condition, but also as a connection of commands in one query (in this case, it is the addition to the query of the command ORDER BY)
Queries INSERT, UPDATE, and DELETE are sent with the connections to the same chart-nodes, but only with appropriate settings.
Simple INSERT looks like this:
INSERT INTO `authors` (`id`, `Name`, `Birthday`) VALUES ('1 ',' Iain Menzies Banks ', '1954-02-16')
UPDATE:
UPDATE `authors` SET `Name` = 'Iain Banks', `Birthday` = '1954-02-17 'WHERE `id` = '1'
DELETE:
DELETE FROM `authors` WHERE `Birthday` <'1954-02-17 '
In the process of construction, the queries using PDO are generated.
With the help of node connections almost any type of queries can be built, including organizing JOIN unions. However, I haven't found the ideal graphic solution for all the complex queries, so I left the opportunity to write the text of a SQL statement in a special node. Looking ahead, I should say that the work with database can be built in the system in different ways, not only with the help of MySQL nodes.
"And Now All Together!"
Finally, the example of combining all three technologies (MySQL, PHP and HTML) in the same chain may look as follows:
Here we have a ready library.tpl template for showing the books from home library. The content is something like that:
<div> <select> <option value=''>all authors</option> %author-list% </select> </div> <br clear="all"/> <div>%book-list%</div>
We have two templates for replacement: %author-list% and %book-list%. In the first one we will have to put option tags, corresponding to all the authors. In the second one we need to put all the books, namely to clone "book block", which contains the HTML code for the output of a book. We need the first list to filter the books by author (for this we will use JQuery function in the template library.tpl).
For the first template %author-list% we add the node Copy ("Author Copy"), which clones the tag "option" according to the data from the table "authors". At the same time it places the name of the author into the tag, and assigns the value of id to "value" attribute. In the output of "Author Copy" node we get the following code:
<option value='1'> Iain Menzies Banks </ option> <option value='2'> Charles Michael Palahniuk </ option> <option value='3'> Mikhail Bulgakov </ option>
"book block" node contains the following code:
<div class='author-%author-id%'> <img width='100' src='images/%image%' align='left' /> <b>%title%</b> by <b>%author%</b> <div>ISBN:%isbn%</div> <small>%description%</small> <br clear='all'/> </div>
The templates in this section are replaced with book data from joined records of two tables "books" and "authors". Table "authors" is here to display the name of the author in the book . The block itself is cloned as many times as we have records in the table "books".
That's what we get as a result:
Conclusion
In this article I have tried to describe the basic idea of the work of the system. In the project there are lots of other possibilities implemented. Some of them are typical for node interfaces. Others I had to invent on my own, taking into account the particular characteristics of the editor. Here is a noninclusive list of the currently available tools:
- Creation of groups — special nodes, inside of which one can hide the whole schemes and create only the required incoming and outgoing connectors. The depth of nesting groups is unlimited.
- Ability to add as a parameter of a group the variables from any scheme that is within this group.
- Saving logic schemes or their parts as snippets for reuse in other scripts.
- Ability to write your own functions that can be displayed in the editor as nodes.
- Creation of your own or import of ready-made classes and the ability to output class methods as nodes.
- Possibility to include other scripts created in the system, and scripts from other files.
- Ability to edit files using the built-in editor.
- Tools for creating and editing MySQL tables.
The result of the work of the editor is a set of generated PHP files. Each scene is a separate script that is saved in a special folder. Once the script has been created, it can be executed without the help of the editor. In addition to that, in the system settings a start-up script can be selected, which includes the main logic of the whole project, based on the data (query string or sent parameters).
The purpose of writing this article is to understand whether such a system could be of interest to developers. I was enthusiastically engaged in this project for too long and from the moment I started working on it my attitude to it has changed. Therefore, I would be most grateful for any feedback and comments, for any questions and suggestions, as well as for participating in testing.
At the moment the project is almost complete and now it is in the process of debugging. Very soon I will be ready to share the beta version. In order to stay informed about the process of development and learn about the release of the project you just need to leave your email address in the appropriate form on the site Mooha.net.
Technical information
In the project the following tools were used:
- HTML5 Canvas and JavaScript (node editor)
- JQuery 2.0 (settings or properties panels)
- Codemirror (built-in file editor)
- CKEditor (advanced editor for database table fields)
- Spectrum Colorpicker (node elements colour change)
- jQuery File Tree Plugin (file system browser)
The server component was tested in the following conditions:
- Apache 2.2.22
- PHP 5.4.3
- MySQL 5.5.24
Subscribe to:
Posts (Atom)