I've been implementing the new Module system in Clay and using it to test the documentation generator Doxygen on the source code. I'm changing some of the coding style in Clay. They say a develoeper's political and social values influence how they write code, well, I'm a conservative. I've always tried to keep my code condensed and as small as possible. I'm trying to move away from that and make the code a little "prettier", while also making it easier to read, and documenting it better.
The Module system allows a module developer complete freedom over his/her module. Each module is in reality a sublibrary to a library named Module, so a module's can be access using Clay's \Library() function or by using the Module library. This also allows developers using the modules use them in ways that fit their coding styles. They can be accessed directly or using methods within the Module object (as well as staticly).
The purpose behind the modules is to provide a layer between libraries and applications. A secondary benefit to the separation is the ability for application platforms to share modules. Modules have their own setup classes and a Manager that resolves module dependencies. The manager can determine if a module is installed, what version is installed or available, and any dependencies it may have or another module has on it. When an application is installed, it can invoke the module Manager and tell the manager it needs or no longer needs a module. The manager then sets up a database or whatever is specified in the module's setup class, including a dependence on another module.
Currently I am working on the "privs" branch within the Clay repository. The purpose of the branch is to mature Clay's privilege system, which led me to implement Modules. The first modules that will be included will be User and Security based, but I plan to move much of the ClayCMS library to modules.
I have been regularly regenerating the documentation with Doxygen to track any problems with changes I'm making and to learn what works best for me. I plan to put the Doxygen generated docs up on the web, likely a placeholder on clay-project.com or on a GitHub page, once the privs branch is merged back into master.
"Doxygen is a documentation system for C++, C, Java, Objective-C, Python, IDL (Corba and Microsoft flavors), Fortran, VHDL, PHP, C#, and to some extent D."
I've struggled to find time to write documentation, especially when I'd much prefer to write code instead. Lately I've been trying to get better at writing documentation inside the code. I decided if I used something that would generate documentation from the code, I could at least provide some type of documentation source. It's a start at least.
Doxygen takes comments placed in source code and uses it to produce a formatted reference documentation. I've been testing it out on my development code and it works really well. I have found, of course, that some of the my comments aren't up to Doxygen's standards, so I will have to fix quite a bit of my in-code documentation. One of the plus sides to Doxygen (and I'm not saying other Doc generators don't do this) is it supports for Markdown documents as well. That is a huge plus, actually, because all of the docs I have written so far are in Markdown format. GitHub also uses Markdown, so it works out quite well.
I am currently in the process of performing a very large upgrade to some of the foundational structures within Clay. Many of the features I had planned to hold off for Clay Framework 2.0 are making their way into the current code base. During this process I've gone from chicken-scratch code, I once used, to better formatted code and standardization of a lot of the ways I write code. Nearly all of the source files have been getting formatting changes, so this is a good time to document as much as possible within the code.
I'm getting ready to fill the void of not having a web site dedicated to the Clay Project, as you may have guessed from my logo annoucement earlier. I'm going to start it out simple and grow it as I can. My main concern is content (is king), so I want to have as much documentation as possible. Doxygen will help me fill that void and save some time in the process.
I've made a Clay Project logo, finally.
I plan to have some logo contests in the future for different parts of the project. This one is just to have something. It's growing on me though :)
I originally planned to keep Clay 1.x to only require PHP 5.3+. Unfortunately, with the (re)introduction of Modules into Clay, I am moving ahead with features provided in PHP 5.4, specifically Traits. PHP 5.4 also offers short tag support (without the need to enable it), which is another feature I've been looking forward to using in templates.
I was considering splitting off the new code base and keeping a line for Clay 1.0 and naming the new line 2.0. I'm not going to do that, as of now at least. I can't find a justifiable reason to move ahead to Clay 2.0 when there isn't a sufficient user base to justify maintaining the current line as 1.x. I kind of knew this would happen if I spent too much time on the older code base.
In reality I don't have a user base to cater to right now, so I don't have to worry about developers that need to use PHP 5.3. The way I see it, I can keep moving Clay forward and advancing it, until I gain a user base that will require me to make more conservative changes to the code base and versions. If I keep using the most up to date features in PHP, until I have filled in the features I want in Clay, eventually the user base will catch up to PHP 5.4+ and we'll be ahead of the competition.
I tried to dodge this change, but, like I said before, I can't justify holding back the code base for a user base that doesn't exist yet. I have a feeling the direction Clay is going will get some people's attention soon enough. Plus, I get to play with new stuff ;)
It seems like I always find something else that "needs" to be fixed when I'm supposed to be working on the privileges system. After moving code around most of the weekend and trying to figure out how to make Libraries a little more accessible by Applications, while retaining access restrictions, I have found an answer (I think). It's actually a feature I dropped from Clay a while back.
Clay Modules are placed in the Module library and act as specialized controllers, providing a link between Libraries and Applications. I dropped the feature in the past, because I couldn't find a good way to make a module different from a Library, but without the functionality of an Application. While browsing through the different libraries that make up the current privilege system, I realized there are a lot of database queries going on in those libraries that depend solely on an Application to create the database backend. That is counterintuitive to the purpose I had set out for libraries. Libraries are meant to be generic or specific, but should be depended on by Applications, not the other way around.
A couple of years makes a difference, I guess. The new Module System will be self-contained within a Library that stores all of the modules and their related classes. The modules will have setup classes that install them and resolve dependencies between them automatically, when initiated within an Application setup class. A Module Library will be used to load module objects and allow an Application to control when and how it is used. The idea is to augment Application functionality, while moving the data backend dependencies away from Applications.
This Module approach seems to be more in-line with what I had originally wanted Libraries to do. A few years ago I realized the Library dependencies on Applications was the opposite of what I was wanting and created Modules. Back then, though, I hadn't created the Application Setup class I have now, and had no idea of how to streamline from Library to Module to Application, without blurring the lines somewhere in the middle. I thought it was pointless to have Libraries and Modules that depended on Applications, instead of the dependencies flowing downstream, so I dropped the Module idea.
I'm still testing it out and trying to work out how to track dependencies. The bright side to that is I haven't worked out Application dependencies and now I can have a proving ground for that. I'll keep you updated and begin pushing some of the changes into the Clay repo soon. Unfortunately all of this has been done in the Clay privs branch, but maybe that'll entice me to finish the privilege system before merging it all back into the master branch. I expect to push quite a bit of the modules into the Clay Framework repo as well, but most of the work will be done from the Clay repo for easier testing.
I've merged the branches and deleted the old cbd2 branch. Everything is tested and working. The upgraded ClayDB's version from 1.92 to 1.96. I will continue development of ClayDB in its repo and merge it into the other repos as more adapters and changes are added.
I will be working on the privileges system again and will likely create a new branch for that. I'm hoping that wont take as long as the ClayDB 2 upgrade did, but its hard to tell how much time I will have from day to day.
By the way, all of the (stable) ClayCMS applications install correctly with the new ClayDB specs. I don't know if you saw how limited tha Data Dictionary was before, but this upgrade is a huge improvement.
Less than 20 points separate the Top 3 teams in the Preseason Poll.
- LSU (18) 1403 pts
- Alabama (20) 1399 pts
- So. Cal (19) 1388 pts
LSU edged out the #1 spot, despite having fewer 1st place votes (18) than both Alabama and Southern California. So. Cal has been been an early favorite to win the Championship this year and are obviously at the top of a few people's lists. 5 of the Top 10 teams are from the SEC, who has 7 of the Top 25 teams in the Poll, which is rounded out with Auburn at #25.
Alabama plays #8 Michigan on Sept 5.
During the ClayDB 2 upgrade I've come to the conclusion that doing the individual library updates within the main Clay or Clay Framework repos really interrupts the development flow. I don't like to think about libraries being ahead of the main code line, but I've lost a lot of ClayCMS development time by doing the ClayDB 2 upgrade within the Clay repo. I had considered switching back and forth between branches, but then I risk losing something I forgot to commit.
The upside to splitting the libraries' development from the main code line is I can now offer them as standalone projects. I can also develop and test the code more thoroughly before beginning any updates by dependent libraries or applications. I have been wanting to offer some of the Clay libraries on PHPClasses.org, so this also gives me the opportunity to do that as well.
Here's the link to the Clay Project on GitHub: https://github.com/organizations/clay. The Clay Project will remain as the parent project to any subprojects I start.
I've pushed the completed prototype for the new Data Dictionary for ClayDB 2, PDO MySQL Data Dictionary. There are still many comments and examples to add, but I can now move on from prototyping a single data dictionary to implementing data dictionaries for other databases. SQLite or PostgreSQL will be next. I may even work on them at the same time.
I still have to update all of the apps to be compatibile with ClayDB 2. Once all of that is completed I will merge back into the master branch and get back to ClayCMS development (among other things :). It's been fun, but I'm tired of reading about databases... Next time something like this will probably be done in a separate repo, that way whatever I was working on before doesn't have to grind to a halt.
A few days ago I blogged about Clay Data Transport (CDaT) and OData. After some consideration and starting the early planning stage for CDaT, I've decided to go a slightly different route.Passage
Passage is the new name for CDaT and is another project within the Clay Project. It will be developed as a library in its own repository on GitHub [https://github.com/clay/passage]. The README in the project repo's index directory explains it, but I'll explain it here as well. Passage is a transaction server for moving data from one source to another. The abstract of the idea is you have web sites that act as nodes that create and receive data, while another web site or server is used as a hub to connect the nodes. There can be multiple hubs, nodes that also act as hubs, and nodes grouped and connected together under groups of hubs.Hubs
Hubs are the router for the data to flow through the nodes and even treat other Hubs as Nodes. They track requests for data access and once data is received they pass on that data to the requestor. Data requests have to be approved by the Node that is providing the data. The Hubs use routes to transport the data based on requests from Nodes. Hubs can route data based on tiered access levels, from all data coming from a node to a single transaction reference.Nodes
Nodes are servers that can act as clients or data sources, even both. They authenticate requests for data, whether it is a request to send or receive. Nodes can not communicate without a Hub and are not required to treat data the same way. The nodes are required to be able to place context on the data they receive and to assign a data type to data they send.Transactions
Each transaction between nodes is recorded by the Hub for reference. The Hub responds to sent data by sending the sender a reference transaction id and then attaches that id to data it sends down stream. Nodes can change the data and resend it, so any Node using that transaction always has the current data. Additional transactions can reference a transaction id, which places attachments to the parent transaction. The Nodes are required to provide association between the data they send or receive an any assigned transaction ids.Data
The data transported by Passage uses data types to provide context. The Hub has no way to identify data as anything other than its data type, it does not place context on the data. The Nodes are required to be able to identify what kind of data they are sending by assigning it a data type. The Nodes are then required to assign context to data types they receive and treat it as needed.
A standard for identifying data within context is currently under development named OContext. The purpose of OContext is to standardize data types for transport and allowing the Node to identify what the data it receives is. The Nodes use a data dictionary to translate data types into how to use the data. Each Node can translate the data differently.Transaction Queues
Nodes and Hubs will have the capability to queue transactions for later use. If a Hub tries unsuccessfully to push data out to a Node or if a Node is not able to send data to a Hub the data will be queued. Nodes can also use a Check-in/Check-out system for data transport. What this allows is for the Node to send data (check-in) when desired and to receive data (check-out) when able. The Hub will know which transactions have occurred and duplicate transactions are avoided. If there is no data in a queue, the Hub can route a request for a Node to check-in transactions for another Node to receive.
That is the basic idea, without going too much into implementation. It is a very flexible data transaction system. Passage is also going to be built so it does not have prerequisite libraries from other platforms. It will be modular. The idea is I can use Passage in Clay and someone else can use Passage in the Zend Framework or in Drupal, without causing any crossover. Many of the features will be built into modules, but it will also allow a Bring Your Own Component approach, so someone using Zend can use Zend libraries in place of a built-in component within Passage.
I've been looking for better ways to manage the development of Clay, ways that may lure more developers to us. Based on the advice from Cal Evans' blog [http://blog.calevans.com/], I've decided to give phpcloud.com a try. It's a cloud environment provided by Zend for developers and it's free, sounds good so far.
I'm setting it up now and will post an update when I've tried it out a little
Update: PHPCloud.com looks to be an excellent service, but it still has some issues that are hard for me to work around. It is currently a "Technology Preview". I'll keep the account, as a do really see a lot of promise in it, but for now I'll have to try something else.