Internal system – Toolkit

I have been using a web based product called ActiveCollab for sometime now …

https://activecollab.com

I have been using ActiveCollab since version 4. When ActiveCollab 5 was introduced it had a complete redesign, less features and a more slick user interface.

More features have been introduced to ActiveCollab 5 though since it was first released.

ActiveCollab is a project management system that also handles Time Tracking and Invoicing plus more.

I will continue using ActiveCollab for projects / task management however for other areas of my business I have created my own internal tool which I have named “Toolkit”.

I built my own system because I wanted a system that would do the following …

Estimate earnings
Fixed fee and retainer projects are entered into the Toolkit. The Toolkit can then estimate monthly and yearly earnings based on the projects entered. The projects also includes a way for me to manage all the projects I am working on.

Incomings
The Toolkit has an area in which all incomings can be entered. The incomings can then be exported in a format appropriate for my accountant.

Outgoings
The Toolkit has an area in which all outgoings can be entered. The outgoings can then be exported in a format appropriate for my accountant.

Mileage
The Toolkit has an area in which all mileages can be entered. The mileages can then be exported in a format appropriate for my accountant.

Leads
The Toolkit has an area in which leads can be managed. I actually pulled this area out of the Neptune – Lead Tracker – http://www.neptuneleadtracker.com software system.

CRM
The Toolkit includes an area in which all the companies and people I have worked for can be managed. ActiveCollab does actually include this feature but the Toolkit version allows more advanced search features and has fields on the create / update forms to match my exact needs.

Timetracking
The Toolkit includes an area in which all time entries can be tracked. ActiveCollab does have this feature.

Invoicing
The Toolkit includes an area in which invoices can be created. Invoices can also be generated from time entries. ActiveCollab does include this feature but the Toolkit version matches my exact needs precisely.

Notes
The Toolkit includes an area to keep notes. ActiveCollab does include this feature.

Files
The Toolkit includes an area to upload files. ActiveCollab does include this feature.

ActiveCollab has a more slick, beautiful user interface but in terms of functionality built specifically for myself obviously the Toolkit wins.

The Toolkit is primarily made up of CRUD screens, no fancy UI / Ajax heavy screens here. Just datagrids and buttons.

I am in the process of live testing the Toolkit I have built and will be moving various functions of my business over to the Toolkit.

Work and not work

An event jogged my memory a couple of days ago and provided the inspiration for this blog article. A number of years ago whilst freelancing I rang a web agency to see if they hired freelancers and whether they had a project I could work on.

I told the person in the conversation about my education, my previous work, my company but I also mentioned that I study programming as a hobby also. Immedietly in a raised voice “we are not doing this as a hobby, we do this as a profession, this is not just some hobby!”. At the time I was shocked, what kind of response is this?

I came to the conclusion that this person obviously cannot conceive that programming can be studied as a profession “and” as a hobby i.e. studied in spare time outside of working hours. The reason the person cannot conceive of this is because this is something the person has never done. This person has never worked on a programming project in his spare time outside of work.

The idea that someone could dedicate extra time outside of professional work to programming something they enjoy is inconceivable to him because he does not enjoy it. Surely anybody who enjoys programming and the greatest programmers out there will know that the programmers who also study programming as a hobby are more likely to be more committed to programming than those who only do it for their profession.

This sort of response can be seen across the whole working spectrum, the more the person takes the work “seriously” the less he is interested in doing work. He takes work “seriously” because he does not really want to work, he gets no enjoyment from doing the work so he cannot conceive that anyone else can enjoy doing work.

In reality he does not work take work seriously at all … “we are not doing this as a hobby, we do this as a profession, this is not just some hobby!” has no real meaning. It says nothing as to the quality of the work he is producing, supposing my code as a hobby is of a greater quality than his code as a profession? His statement says nothing, it is just an empty statement stated with the tone of someone trying to be serious but cannot be taken seriously.

Another situation might be the boss who does not let his employees work from home. This can be well rationalized “we do not like our code on other peoples computers it is a security risk” etc. In reality the boss does not let his employees work from home because the boss cannot concieve of an employee who can actually do work when the boss is not watching. You have to be there so the boss can see that the work is being done.

Again this ties into the enjoyment aspect of it. You cannot possibly enjoy work that much that you would go home and work when he is not watching because he does not enjoy doing the work that much and it is not something he would do. It is the joy aspect that is being targetted.

There have been a couple of occasions in the past where I have been working on a programming task in an office and I have looked up to notice a collegue staring at me. One of the collegues was staring at me in joy with a smile on his face, one of the collegues was also staring at me in joy with a smile on his face and he made the statement “you thought you where being clever there didnt you”.

I never thought much of this at the time but the more I thought about it the more it made sense, in those collegues eyes I must have looked similar to this …

https://www.youtube.com/watch?v=laG5XawvtqA&t=980

No I am not claiming to be some super genius, but what they saw was probably something similar to that in their own eyes. What they where actually seeing was the “aliveness” of myself actually enjoying working on a programming task.

This can also be seen when asked to struggle with a programming task. Say a new developer is being introduced to a new codebase …

Myself …

This is how I implemented file uploads.
This is how I implemented error handling.
I put these methods in this Model here and they do this and help with that.
I created this helper class to deal with this.

General developer in the industry …

Yea, just look through the code and struggle with it, mate.

A little bit of struggle is not wrong. Sometimes I already more-a-less knew the answer I was asking I just needed confirmation. Sometimes the struggle can create the “fire” to actually do the work.

I am not saying there should be no struggle I am just saying there should be a balance. In this industry I see far more of the “struggle” path than the opposite, even to the point of asking a question no longer gets an answer but the developer is expected to struggle for 2 hours to get an answer to a question that would have taken 30 seconds to answer. I mean how is this sort of struggle productive? In my opinion, it is counter productive.

The “struggle” path developer rationalization to this is always, “well if I had given you the answer I may as well of been doing the work for you”. Yes in some situations this could be valid but in a lot of the situations I have encountered this it was clearly not valid. It is just an excuse for the “struggle” path developers own lack of effort.

In my own opinion all questions should be answered no matter how simple the answer may be. It may not be simple for the developer who is just starting out. Answering the question helps relieve a blockage in the production.

The next arguement could be “how I am suppose to get my own work done if I am answering questions all day” this could be valid, but then it is a management problem, the manager should account for the fact that the more settled in developers will need to answer the new developers questions.

I personally think all “lead developers” job roles should consist of helping / answering developers questions and maintaining the consistency of the codebase / architecture of the code.

That is all the lead developer needs to be doing, the lead developer should act as a “leader” to the none lead developers and as a person who ensures the cohesion of the codebase as a whole releaving blockages in the production by helping the none lead developers.

The lead developer should not have a huge number (if any) tasks on his plate at all and his / her sole focus should be on the none lead developers, leading them. In reality what generally happens is the lead developer has just as much work as the none lead developers have to do.

What happens? No help is given to the none lead developers, tasks take longer by not getting answers to questions, none lead developers look incompetent, lead developers looks good because his work is getting done and generally everything just takes longer.

Developer asks a lead developer a question, answer it. Developer is stuck on writing a block of code, pair programmme with the developer for 30 minutes, relieve the blockage. Lead developer checks the code commits for the day and finds mistakes, point out the mistakes and ask the developer to correct them. Lead developer checks the code commits for the day and finds inconsistencies in the code which would not fit well with the rest of the code, architectural inconsistencies, point out the inconsistencies and ask the developer to amend them.

The lead developers role in my ideal basically consists of leading and empowering the none lead developers. In most companies the lead developer is merely the developer who has been at the company the longest, not really leading anyone at all. The lead developer does the same job as the none lead developers and to some extent is expected to do more than the none lead developers.

What is worse is that the lead developer is given “first dibs” on most projects, this means the lead developers code 90% of the codebase and then it is passed onto the none lead developers to amend and maintain.

Manager …

Ohh I will give Lead Developer X this project to do, like I did with the last one and the one before that because he knows what he is doing and he gets the work done within the deadlines. Of course he knows what he is doing, he started from a blank canvas. He makes the manager look good.

Once the project completed and passed on the lead developer is then working on his next project which he got “first dibs” on, so he is now “too busy” to answer the questions of the none lead developers even though he is the only person who can really answer the questions as he wrote most of the code in the whole business.

This kind of position is somewhat of a nice setup for a lead developer, even though the lead developer may be writing more code than the average none lead developer.

A solution? Each developer is given his / her own project or set of projects to build and maintain. If the project is absolutely a huge project that requires multiple developers, then the lead developer is the “leader” and the none lead developers are the “developers” doing most of the work.

Is University important?

There are lots and lots of developers in the industry who never went to University and some never went to College either. Well I went to both University and College (Sixth Form) and in my opinion it was definitly worth it.

To start with before University I never had any interest in programming and I would for the most part say it is only because of University that I became a programmer. At University I was given a series of lectures and practical lessons. The lectures and practical lessons form the equivalent of a part time week. In the other half of the week you are expected to be self learning the subject discussed in the lectures and practical lessons.

None of this is mandatory, you do not have to turn up to lectures or practical lessons but seen as though you are paying for the lectures and practical lessons it would be a waste of money not to. University treated me more like an adult than any other educational organisation I had been to prior to University. You are actually a customer of the University.

A certain opinion on University in terms of the programming world are that Universities do not prepare you for the “real world” of development. To some extent this is true, University certainly gives you the ideal programming world. A world in which your profession is respected and you are given ample time to conduct your research.

I once worked for a company where one of the developers stated on a task I was working on “its not rocket science is it”. For the most part the developer was quite right, it was not the hardest task in the world to be working on. However that is besides the point, what does it say when a developer makes such a comment? This developer degrades his own profession by making the claim that the work is simple and easy.

You would not hear such comments at University as far as I am aware because the lecturers respect their profession and they create that same feeling of respect in their own students. You are a computer scientist working to solve real world problems.

A lot of companies are formed by people who never went to University and a lot of companies have developers working at the company that never went to University. For anyone who preaches “University does not prepare students for the real world of development” this is probably the reason why.

The people at the above companies never went to University and never had that same sense of respect instilled in them for their profession. Let us not degrade our own profession by debasing ourselves and making comments like everything is simple and easy, even if it is. In other words the degraded “real world” of development is at odds with the Universities respectable “real world” of development or rather there are so many people joining this bandwagon that it has somewhat become the “real world” because of themselves.

When you go to a University you have the fundamentals of programming explained to you in the lectures. The correct terminology is used to explain everything and it is explained in a professional manner. The feeling this creates in you makes you want to be apart of the latest innovations, discoveries and technologies.

Looking back after a number of years in development I would say that University teaches the foundations of each subject on the course you have taken on. I would agree that there is a lot of self learning to do in your own time but that is to be expected of University is it not? Maybe the people who are saying “University does not prepare for the real world” have been dealing with people who never did any self learning in their free time whilst at University.

Looking back there was a lot of people who only seemed to go to the lectures and practical lessons when they could be bothered. Never seeming to learn anything outside of the lectures and practical lessons. It is quite possible that they still managed to scrape a pass on their degree and maybe that is the reason for the skewed perception.

I would argue that the people who did that are probably in the same category as those people who could not be bothered to go to University at all. They are probably the same people now complaining that “University does not prepare you for the real world”. Ultimately though everyone is a individual whether they went to University or not and it is all about the investment they have made in the programming studies.

I think University does introduce programming students to some concepts they may not have readily picked up outside of going to University. Take “recursion” for example. I have worked on codebases that had parts of code written in ways that could have really used recursion and should have used recursion.

It seemed to me that the programmer had struggled to write the piece of code because he was unfamiliar with the concept of recursion. Not that he could not do recursion but that he was not even aware of recursions existence. This would have been covered at University so at least the programmer would know recursion existed. Had the programmer known recursion existed it would have most likely been used.

University gives you the ideal development world view and I assume the very top companies in the programming world share similarities with this world view. I can only assume this is the case because it all depends on the people working for the said companies.

So was University important? From the learning programming standpoint, perhapes not so much. However from instilling respect in your chosen profession, very much so.

King of Adem – progress update – building queue

Progress has been made on the King of Adem project. For those of you not familiar with this project please read the original King of Adem post at …

http://www.jamesbarnsley.com/site/2017/08/13/king-of-adem/

Since the update I have been primarily working on the building queue. The building queue is the queue which buildings get inserted into when a player wishes to construct a building in one of his / her regions.

To get to the construct building screen the user has to be in one of his / her regions so those areas to get to that screen had to be part built.

The areas part built are namely user, game, region and building. Once on the construct building screen the user can select from a list of buildings that can be constructed.

Each building is validated on a case by case basis and if it does not pass the validations the building cannot be constructed, here are some of the validations so far …

Check fund available
The player must have the correct amount of fund available to construct the building. The amount of funds available is the players start of turn funds subtracting any funds that have been used in the current turn.

For example player may have started with 1000 coins, if a player constructs a building for 300 coins then the program gets the players start of turn funds but substracts 300 coins because it knows a building worth 300 coins was requested to be constructed this current turn.

Check building queue full
The building queue may only allow so many buildings to be in the queue at any one time for each region. Currently this number is set at 5 and the number is stored in a config file.

Check building already built
Check if the building has already been constructed in the region. Each building has a “constructionLimitPerRegion” variable which stores the number of times that particular building may be constructed in each region.

Check dependent buildings constructed
Each building can have what are called dependent buildings. These are buildings which need to be constructed first before you can build the current building.

Also the validation knows whether a building is an upgrade of an existing building or a completely different building that just happens to depend on another building being constructed.

All kinds of checking had to be implemented for this validation, whether building is an upgrade / dependant or just dependant, whether advanced tier buildings already exist for lower dependant buildings, whether lower dependant buildings already exist etc.

Check advanced tier building exists
Check if an advanced tier building already exists for the current building wanting to be constructed. If the advanced tier building already exists then there is no reason to build the current building.

Again this needs to take into account that some buildings can be constructed more than once so in some instances it may need to allow the building to be constructed even though the lower building has already been constructed.

The above validations are run everytime a user issues the command to insert a building into the building queue for a particular region. These validations are run from the “Building” Model. Each building is also given its own Model, the hierachy is like as follows …

\yii\db\ActiveRecord -> \app\components\ActiveRecord -> \app\models\Building -> \app\models\BuildingTownBarracks

Or …

\yii\db\ActiveRecord -> \app\components\ActiveRecord -> \app\models\Building -> \app\models\BuildingPort

etc.

I can run special validations for specific buildings by placing those validations in the specific building class. Those validations will only run for the specific building and not all buildings.

That is as far as I have got so far. Still a long, long way to go, more updates to come soon.

King of Adem

The latest personal project I am working on which also happens to be the largest personal project I have worked on so far.

King of Adem is a game for tabletop minature wargamers. Players manage their kingdom through the app. Kingdoms consist of a number of a territories, each player starts with one.

In each territory there is a capital city in which buildings can be constructed. Buildings provide benefits like more money or the ability to recruit armies.

When 2 armies collide by moving into each other the players can then resolve the battle as normal by playing their normal game of tabletop minatures.

The game takes place on an island called Adem. Adem is a recently discovered island where each player is fighting for control. You fight to become the “King of Adem” by conquering the whole island.

Although Adem is stated as “recently” discovered, this only means “recently” in the grand scheme of things. Adem actually has a number of Kings / Factions already on the island and each Kingdom has some infrastructure.

The project is currently divided into 3 parts …

King of Adem Diagram

Part 1 – Game Server
The Game Server is currently in construction and is being developed using the Yii 2 framework written in PHP. The database is MySQL. The server takes view requests and command requests from the client application.

View requests provide data to the client in JSON format. This data has everything needed to render views for the game for example, building, army, territory data.

Command requests provide data to the server about interactions happening in the client application. For example move commands, construct building commands etc.

The decision to use View / Command requests was chosen instead of opening the server up entirely.

Essentially one option could have been for each Data model on the server to have a view, insert, update, delete request to use for either commands, views or both.

This would give the client more freedom at the cost of putting more logic onto the client.

As the first client is being developed using IOS and IOS uses a view per screen type of structure then View requests make sense. Giving the client the exact data the client needs in order to render the view.

Part 2 – Game Test Client
The Game Test Client is currently being developed alongside the Game Server and serves as a testing ground for the development of the server.

The Game Test Client can send View requests to the server and read / present the data from the server in a user friendly way. The Game Test Client can send command requests to the server.

The Game Test Client is being developed using the Yii 2 framework written in PHP. The Game Test Client is my own personal client for testing the server and is not the actual client the end user / player will be using.

Part 3 – IOS Game Client
The IOS Game Client is the actual client the end user / player will be using to play King of Adem. The IOS Game Client as you might have guessed will be an IPhone application. There may also be the ability to run the application on the IPad as well.

I decided to develop the IOS Game Client because I wanted the game to be portable and it cannot get much more portable than having the game on your phone.

I also choose the game to be an IOS application because I want to improve my IOS skills and offer IOS development as part of the everyday services I offer to my clients.

Once these 3 parts are complete the game will be playable and released to the public. Part 1 and 2 are already in construction, Part 3 will be constructed after parts 1 and 2 and completed.

After the game has been released to the public, a number of other client applications could potentially be developed including an Android version and a full game client developed in Unity3D or similar.

The IOS client will be more management like as opposed to a graphical game. The IOS application could include graphics etc, but it will not be at the level of a full game like experience. This is why I suggest a later developed Unity3D or similar game engine development.

I will be keeping everyone informed of the development of this game through the blog so keep checking back for further updates.

Yii / Yii2 checking controller params at the access control level

One of the areas I notice in a lot of Yii / Yii 2 web applications is the checking of parameter variables in what I consider the wrong areas.

Here is an example of one of my controller methods for saving a Book …

So what stands out about this method? No load Model method? No checking to see whether the Book has a valid ID? How will we know whether the Book has a valid ID, how will we know whether the Book is suppose to be accessed by the particular user that is logged into the system?

The answer is, the access rules and bizrules of the web software application. All controller methods should be assigned to an access rule. If the controller method contains parameters then the controller method should be assigned to an access rule as well as a bizrule.

For smaller / less complex applications you can actually emulate the bizrule by using “matchCallback” which is spoken about in this article I wrote previously …

http://www.jamesbarnsley.com/site/2017/06/13/yii-2-user-permission-using-matchcallback/

You can also read my previous article on what I think of the Yii / Yii2 load Model method …

http://www.jamesbarnsley.com/site/2015/04/22/yii-loadmodel-method/

All controller parameters should be checked and validated in the bizrule, by the time the parameter reaches the controller method there should no need to have any checking in the controller method at all for the controller parameters.

If the Book does not exist or the Book is not accessible by the current logged in user then that will be handled before any controller method code gets executed in the first place.

There is no reason to have this level of checking in the controller method itself. Doing it the way I have described also keeps the controller more clean as extra code is not needed to check the controller parameters.

Solution for using DatePickers on forms in Yii2

Sometimes in web software applications we need to collect dates such as a start date or end date using forms. In this article I will be using the Kartik DatePicker widget which can be found here …

http://demos.krajee.com/widget-details/datepicker

For the purposes of this article a master Model will need to be created. I discussed how to create master Models and Controllers in the following article …

Create a master Model and Controller in Yii2

So based on the master Model now being created the article can continue. Add the field to the form using code similar to this. Include the plugin in the form …

Add the field to the form …

In the master Model 2 methods needed to be created …

And …

These methods can be called to save the “date” data to the database from any of the child Models. The best place to call the methods would be in the “beforeSave” and “afterFind” methods as follows …

And …

Depending on how you have setup the validations for the “date” attribute you may need to run the “saveDateFormat” and “dateFormat” methods in the “beforeValidate” and “afterValidate” methods as well as the “beforeSave” and “afterFind”.

This depends on whether your validations are formatting the date as the date would be saved to the database or validating against the format the date appears on the form.

If your validations validate against the format the date saves to the database then calling “saveDateFormat” and “dateFormat” methods in the “beforeValidate” and “afterValidate” methods will not be needed.

I generally setup my validations so I do not need to run the “saveDateFormat” and “dateFormat” methods in the “beforeValidate” and “afterValidate” methods only through the “beforeSave” and “afterFind” methods.

Using the above code you can implement multiple datepickers on the same form or across different forms. Follow the same logic, add the field or fields to the form and call the “saveDateFormat” and “dateFormat” methods from the child Model.

Yii 2 User Permission using matchCallback

As a web software developer I sometimes need to implement some kind of user permissions in the web applications that I build. The Yii 2 framework which I currently use has a built in system for doing just this however for applications that need a more lightweight user permission system the Yii 2 framework has introduced a method which was not available in the Yii 1 framework that can be used to easily build lightweight user permissions and that method is “matchCallback”.

Both Yii 1 and Yii 2 have a user permissions system named RBAC. In Yii 1 RBAC is implemented by writing code to define the user permissions, this code then populates the database tables with the user permissions. The user permissions are then applied to controller methods via more code known as access rules.

User permissions can also have what is known as a “bizRule”, a “bizRule” is an additional piece of code executed with the user permission that needs to evaluate to true in order for the user permission to be true. This can be used to implement checks to see if the record being displayed on the page is owned by the user trying to access the page as well as various other checks. The “bizRule” code is stored in the database along with the user permission.

One of the differences between the Yii 2 implementation of RBAC and the Yii 1 version is that the Yii 2 “bizRule” code is stored in code files and not the database. This makes more sense to me as the “bizRule” code is kept alongside the other code in the frameworks, models, controllers, views etc.

Infact it makes so much sense that during the later stages of using the Yii 1 framework I actually just stored class / function references in the “bizRule” code and wrote my real “bizRule” alongside the rest of framework code. Essentially implementing my own version of what Yii 2 covers as standard.

As mentioned above Yii 2 now implements a method called “matchCallback”. This means for web applications requiring simple user permissions RBAC is not needed as the “matchCallback” can be used to define the “bizRule” directly in the access rules …

So as shown above for web applications only requiring simple user permissions, the access rules are all that are needed.

array_map plus other possibilities

When building web applications certain patterns appear in our code. A pattern that use to appear in my code quite regularly is shown in the example below …

Essentially we have an array of data and we want to create a new array out of only part of the original data. The above code can be abstracted using the PHP array_map function as shown in the example below …

PHP includes a number of these functions within its library. PHP also gives you the ability to create your own. The above array_map function can be created in PHP as follows …

Using code similar to the above is also the key to implementing iteration functions on collections allowing you to write code as such …

Or filtering …

The possibilities this opens are numerous. If you would like to know more about writing code similar to the above then I can recommend the following book …

https://adamwathan.me/refactoring-to-collections/

Refactoring to Collections – The Definitive Guide to Curing the Common Loop by Adam Wathan

Select2 library solution in Yii2 part 4 – retrieving the data

To continue on from the previous article I wrote about saving the Select2 data …

http://www.jamesbarnsley.com/site/2016/11/07/select2-library-solution-in-yii2-part-3-saving-the-data/

I will now talk about how to the retrieve the stored Select2 data so that the data can be displayed on the front-end Select2 field. Following on from the Task / Countries example, place the following “loadSelect2Items” method inside the Master model or Task model if you did not create a Master model …

The code above can be explained as follows …

  1. Line 3 – return the array of Select2 data using the “array_map” function
  2. Line 4 – store the related entity models name into a variable, in this case that would be the Country models name. This uses the Country relationship that was created in “Part 2” of this article series
  3. Line 5 – store the IDs of the related relationship entity model in the return data, in this case that would be the IDs of the Country model records
  4. Line 6 – find all the records for the model / model property in the “select2_data” table, in this case that would be the Task model and “countries” property

The next step is use the “loadSelect2Items” in the Task model to the load the Select2 data into the “countries” property, place the following inside the “afterFind” method of the Task model …

So that concludes the Select2 library solution for Yii2. I have covered the creation of the necessary database table and the read / write methods for reading / writing the data.

After the initial methods have been setup this solution will provide you with a way of easily putting a Select2 field on your forms that allows for many to many relationships between models with just a couple of line of code.