FluidTYPO3: Contribution Guide

Glossary

• Upstream refers to our original repository location.
• Fork refers to your forked copy of the upstream.
• Remote refers to the remote location of either upstream or origin.
• Origin refers to the specific remote that is the location of your fork.
• Pull request is a special type of issue you file on GitHub to request merging of your changes from your branch to ours.
• Continuous integration refers to the process of automatically building and integrating code changes.
• Build refers to one execution of a continous integration test, for example testing your changes against PHP version 5.5 and TYPO3 6.2.

Types of contributions

Contributions usually fall into one of two categories:

1. Fixes for problems, smaller changes to correct behaviors, documentation changes.
2. New features.

When your contribution falls into the "New features" category you should start by either creating a GitHub issue or join the IRC channel to discuss your intended feature. We are always happy to discuss new features and the discussions almost always add to the quality/usefulness of the feature.

When your contribution falls into the first category you do not need to open this discussion. You are of course welcome to discuss it anyway, if you have doubts about the implementation and wish to make sure you find the right solution.

The quick and dirty version

If you are not yet comfortable with Git, skip this section and keep reading.

If you are comfortable with Git and GitHub in general, these few pointers should be enough to get you up and running:

• Fork our upstream repository and add a remote named upstream using our repository's URL.
• Never work in the development branch - create working branches for each change you want to make.
• Initialize the repository by running composer install. This creates a ./vendor directory which you can freely remove after your work is done, but which if removed, must be recreated using composer install again before you can once again commit changes.
• Initialise the local testing environment by running ./vendor/bin/make. This establishes hook scripts for Git which cause code style validation and unit tests to be required to pass before a commit can be made.
• Install our CodeSniffer standard (located in ./vendor/fluidtypo3/codingstandards) in your IDE or use ./vendor/bin/checkstyle whenever you want to validate the code style.
• Set up your IDE to execute unit tests using the phpunit.xml.dist file as configuration file. The rest should take care of itself. To manually run tests with the default configuration use ./vendor/bin/runtests or ./vendor/bin/runcoverage to run tests with or without code coverage output.
• When loaded in your IDE, the project root folder should be the repository - not the document root of any site which has the extension installed.
• All development and validation scripts will function without a running TYPO3 site. No database nor file access is required.

Getting the source code locally

The first thing you have to do is to create a fork of the repository you wish to work on - this creates a copy of the repository linked to the upstream but with full permissions for you to create new branches and commits. It is from this fork you create pull requests (which basically are requests to merge one of your branches into our branch).

You then clone that repository instead of the original (the upstream) to your local file system:

git clone https://github.com/$username/$repository.git

Where $username is your GitHub username and $repository is the name of the repository you cloned, for example flux or vhs.

Change directory to the directory that was cloned (example if you cloned the vhs repository):

cd vhs

Once cloned, you add the upstream of the repository as a secondary remote:

git remote add upstream https://github.com/FluidTYPO3/$repository.git

Where $repository is the name of the repository itself, for example flux or vhs.

Having the upstream remote at hand allows you to quickly get all new changes from the development branch on upstream to ensure those changes are used when you create a new working branch for a change.

To keep your local development branch up-to-date:

git checkout development
git pull --rebase upstream development

You can perform this operation as many times as you want, but make sure any changes you have made have been properly committed first.

Initialising the repository

To start using the testing and code style validation features the repository must be initialised with composer:

composer install

This will create a ./vendor directory in the repository root folder - the ./vendor directory is automatically ignored by Git. You can then use the scripts that have been installed into ./vendor/bin. The first script to be executed is the make script which performs an initial validation and test, and installs itself as hook script in Git, ensuring that the make command runs every time you create a new commit.

./vendor/bin/make

The make script is a batch script which runs the checkstyle and runtests scripts. You can run each of those individually before making commits but they must both be executed again (and pass successfully) every time you make a commit.

Also installed is a simple commit message validation hook script which only runs when you finish editing a commit message. This script can be bypassed by using the -m "commit message text" parameter for Git but bypassing it is not recommended. The hook script checks that your commit message starts with a valid prefix and that it uses proper formatting. This hook script runs before the make script when you create new commits.

You are required to use the shipped validation tools. This helps remove most of the issues regarding coding style and errors (when the code is unit tested) and makes the review process much easier.

Creating a work branch

The second thing to do after having cloned the repository is to create a local working branch so you do not commit changes to the development branch (this is likely to cause problems if you reuse your fork for your next change).

git checkout -b mycoolfeature development

This causes a new branch called mycoolfeature to be created and checked out, and the HEAD of that branch to be set to the current development branch's most recent commit.

You should do this even when the change you wish to make is a simple one-line change. Keeping the development branch clean is vital - it serves as the starting point for all changes you make in the future.

Making your changes

After your new branch is checked out you can make your changes. You then commit them to Git:

git add $file
git commit

Where $file is either a single filename, a list of filenames separated by spaces or a shell path expression such as * or *.php.

Git will then ask you to fill in a message. We have a few simple rules for the formatting of this message:

1. The commit must start with a valid prefix to help us identify the nature of the change. Valid prefixes are:
   1. [BUGFIX] when the commit fixes a bug
   2. [FEATURE] when the commit adds new capabilities
   3. [DOC] when the commit only touches documentation (README, PHP doc comments, etc)
   4. [TASK] when the commit does not fall into any of the other categories
2. After the prefix the commit subject must start with an uppercase letter, e.g. [BUGFIX] Fixed problem abc with class xyz.

Save and close the message once you are finished.

The shipped validation and testing scripts will now run. This can take anywhere from just a few seconds to around a minute - if you run Git from the command line you will see continuous feedback, but if you use a GUI for Git you may not see any feedback at all until the scripts are done, at which point the GUI will either display an error output or continue.

If the checks fail

If your changes introduced failures in unit tests or does not comply with the coding style, an error message will be presented telling you exactly what went wrong. To handle the two types of errors:

1. If you experience an error in coding style there are two ways to fix it:
   1. You check the output for file name and line numbers and manually correct the reported problems.
   2. Or you can call ./vendor/bin/phpcbf which will attempt to fix errors that can be fixed (and report any that cannot). Errors that cannot be fixed automatically must of course be manually fixed.
2. If you experience a failing unit test, consider whether:
   1. Your change maybe should include a change to the unit test because your change introduces a different behavior which tests no longer cover. If this is the case then the unit tests must be changed (and you are welcome to contact the team for assistance with this!).
   2. Your change might contain actual errors such as incorrectly defined variables, wrong function names or incorrect conditions. If the code contains such errors, naturally they must be fixed.

To confirm that you have solved the issues with code style you can run the checkstyle command:

./vendor/bin/checkstyle

And to confirm that your changes comply with the unit tests:

./vendor/bin/runtests

When your change passes both checks you can amend the commit you made, adding to it the new changes you made in order to pass the checks:

git commit -a --amend

The shipped validation scripts will now run again. Repeat as many times as necessary until all checks are passing.

More experienced users may want to simply amend the commit to trigger both checks.

If the checks pass

If both checks pass your commit contains no formal errors and causes none of the tests to fail. This means it is ready to be published to origin - to your fork:

If the working branch has not yet been published, publish it:

git push origin $nameofbranch -u

Where $nameofbranch is the name you want to use for the branch on your origin - usually you would use the same name as your local branch.

The -u parameter is there to update tracking, meaning that your branch will automatically push to and pull from this branch after that.

If the working branch was previously published, forcefully update it:

git push -f

Git may ask for your credentials when pushing. You can avoid this by adding an SSH key to your GitHub account and replacing https:// in all GitHub URLs with a ssh://.

If the push command fails with an error, consult the GitHub documentation to find out what may be wrong. In many cases the problem is a typo in the repository URL or credentials.

If the push command succeeds you are now ready to create the pull request through GitHub.

Creating a pull request

To create the pull request which informs the team that you wish to have changes from your branch merged into ours, start by browsing to the URL of your fork on GitHub, for example:

https://github.com/$username/$repository

Where $username is your own username for GitHub and $repository is the name of the repository you forked, for example vhs or flux.

Assuming that you published the branch recently, GitHub will present you with a highlighted option to quickly create a pull request from your branch to ours. This highligted option is presented as one line and a small green button at the top of the view, with one line per branch you pushed (if you created more than one change).

Press the create pull request button to open a preconfigured form that creates the pull request. In this form, one of two things can happen:

1. If you have only a single commit in your branch that is not in ours, GitHub will read the commit message from it and prefill the form, in which case you can go right ahead and create the pull request.
2. If you have more than one commit in your branch, GitHub will not prefill the subject and body fields. In this case you must (at least briefly) describe what your changes will do and why they were made.

The better you describe your change and the reasons for it the more likely it is that it will be accepted.

If your pull request also solves a reported issue you can note so in the pull request to have it linked to that issue. An example pull request body (where the first line is the subject):

[FEATURE] Made VHS more awesome

This change makes VHS more awesome by increasing the
awesome factor.

Close: #123

Where 123 is the issue ID (of an issue in the same repository - otherwise, use the full URL).

You can reference issues using Close, Resolves, Fixes or simply by mentioning the issue number with a hash mark, e.g. #123, or the full URL of an issue anywhere in the text.

When the form is completed, press the big, green Create pull request button. The next thing that happens is that GitHub triggers an action in our continuous integration server (a Jenkins installation) which performs a second level of checks for your changes, among other things running the tests on different versions of TYPO3 and PHP. This build process will either succeed or fail, in which case you will have to correct any issues and update the pull request.

How the pull request is validated

When you create a pull request to any of the FluidTYPO3 repositories the following takes place:

1. An instant-feedback system called Gizzle uses our coding style rules and the same commit message validation used in the Git hook to perform a fast analysis of your changes. Gizzle analyses the source code of only files that you changed in the commits belonging to the pull request. If any problems are detected, Gizzle puts messages in the pull request and/or each line of code you changed, reporting any problems detected by CodeSniffer.
2. Our continuous integration server, https://jenkins.fluidtypo3.org, runs a number of builds to confirm that your changes execute correctly on all versions of TYPO3 that we support, and all versions of PHP supported by that TYPO3 version.

Failures from either of these systems will be reported as very clear status messages in the pull request itself.

Resources involved in our continuous integration concerned with validation:

• NamelessCoder/gizzle provides the basis for near-instant feedback using custom plugins that listen for GitHub changes.
• FluidTYPO3/fluidtypo3-gizzle contains our custom plugins to provide feedback for GitHub changes.
• FluidTYPO3 Jenkins runs our matrix of builds for PHP and TYPO3 version combinations and provides feedback.
• FluidTYPO3 Coveralls archives our unit test coverage statistics and can provide you with feedback about changes in coverage introduced by your pull request.

If the pull request validations fail

Normally, failures reported by the continuous integration system are caused by one of the following and can be solved as follows:

1. The changes included in the pull request was made through the GitHub web interface and were not pre-validated. We require that you "fix" this problem by cloning the repository to your local file system and closely following the steps described in the chapters before - and so allowing the pre-validation to run and catch any problems before they are published. Unfortunately, GitHub's web interface lacks the ability to update commits in a pull request and will instead create new commits which if merged would cause the history to fill with noise.
2. The commits were made locally but the repository was not initialised as required. Do so by following the steps described in the chapters before, then continue to the next chapter.
3. The changes included in the pull request cause one or more unit tests to fail on a version of PHP or TYPO3 (or both) that was not tested by the checking scripts. For example, your code may depend on a TYPO3 feature that does not exist in one or more of our supported versions, or you may have used a PHP function or instruction that does not work on one or more of the PHP versions supported by the TYPO3 versions we support.

This second validation phase covers a lot more ground than the local scripts can and as such, it might reveal some more complicated issues which require our assistance. If this validation fails for a pull request you create, please don't be afraid to ask for help - sooner rather than later. We will then do what we can do make sure the contribution process goes smoothly.

How fix your pull request

Here we have several scenarios that you could have:

A Team member is asking for changes in your commited code: Your commit message is fine and now you need to change something related with the code. Make the changes and then:

git add $file
git commit --amend --no-edit && git push --force

Your code is fine but your commit message is wrong:

git commit --amend -m "Commit message well done" && git push --force

Your code and your commit message are fine but you didn't set properly your username:

git commit --amend --no-edit --author "New Author Name <[email protected]>" && git push --force

If the pull request validations pass

You are nearly there! All that remains is the final, human review of your changes. One or more team members will read through the code and test it, giving their remarks along the way.

Team members may request changes at this point, taking into consideration how the change would affect users of the extension(s) as a whole. If no such changes are necessary and the pull request is approved, your code gets merged and is released to the official development branch instantly - and released to TER when the next release is built (frequency depends on repository, changes queued, compatibility with other extensions and similar concerns).

Team members' role when reviewing

The team members are responsible towards all users and will act accordingly. The team may choose to reject your changes but will always give a detailed response describing why it was rejected and if possible, suggesting alternative ways to implement the changes. Currently we reject approximately 5-10% of pull requests but of those, almost all receive an alternative suggestion along with the rejection (if the current pull request cannot be resolved in any way).

We ask that you give special weight to the team members' requests because:

• The team likely is aware of a greater variety of usages which must be supported.
• The team is responsible for communicating the changes and must be able to justify them.
• The team is responsible for maintaining the solution after you contribute it.

That being said: we always listen to reasonable arguments both for and against the team's decisions.

Dealing with a rejection

If your pull request is rejected there may still be a way to reach your goal. In the cases where a rejection is given with good reason but none of the suggested alternatives would fit your needs (or if no alternatives are given), there are a few things you might be able to do:

• Come talk to us about the goal you have (IRC is a great way to get in touch quickly). Discussing in a more interactive way usually results in some new ideas and alternatives.
• Contribute man-power if the problem is one of maintenance or communication - for example, offer to help maintaining the solution or relieve team members in general, to write articles or documentation about new features, to blog about breaking changes and features, etc.
• Hire one or more team members on a freelance basis, to conceive and/or create a solution that fits both your needs and those of the community.

But most importantly: don't let one or a couple of rejections stop you from contributing!

Extra: Making checks run from your code editor

Using an editor with support for our validation tools can provide you with additional feedback in the editor itself, for example highlighting lines that contain code style problems or run unit tests via a keyboard shortcut.

Depending on which editor (IDE) you use, there are several ways that you can excute the check scripts from within the editor, for example running all unit tests when you press a key combination. All checks are able to run without requiring any database or file system content apart from source code (i.e. TYPO3 temp folders and configuration files) - all they require is initialisation using composer install without the --no-dev parameter.

An incomplete list of ways to integrate:

• If your editor has phpunit support (with or without coverage display) or similar you can create a run configuration which calls ./vendor/bin/phpunit -c phpunit.xml.dist. If your IDE supports coverage display it will automatically be able to append the required arguments.
• If your editor has git support and understands hook scripts or uses the CLI interface, committing files through your IDE will execute the tests but will only provide a text feedback if issues arise.
• If your editor has CodeSniffer support you can enable it and point to the custom rule located in ./vendor/fluidtypo3/coding-standards. This will, depending on the level of IDE integration, provide you with a level of feedback about any coding style violations. Some IDEs can run this automatically, others require triggering it manually.
• If your editor has none of these things but can execute CLI commands you can either manually run the ./vendor/bin/checkstyle and ./vendor/bin/runtests scripts - or you can configure keyboard shortcuts or panel buttons to run them.

It is particularly recommended to use an editor that supports CodeSniffer validation by highlighting or jumping to lines that violate the coding style rules, for example PHPStorm, IntelliJ IDEA, NetBeans, Zend Studio and Eclipse (via a plugin).

Extra: Making shortcuts to check scripts

Typing ./vendor/bin/... can quickly become boring if you have to do it a lot.

If you use a UNIX system like Linux or OSX and frequently run the ./vendor/bin scripts you can extend your PATH variable to look in vendor/bin before looking in global locations. This allows you to run commands like checkstyle and runtest without including the path to the script, e.g. ./vendor/bin/checkstyle. To extend the path variable, set it in for example .bash_profile in your user's home directory:

nano ~/.bash_profile
# add line to file:
export PATH=vendor/bin:$PATH
# "source" the file (run instructions) or log out and back in
source ~/.bash_profile

This will make bash first look in ./vendor/bin when you type a command and will work whenever you are in the repository root of any FluidTYPO3 repository. The commands that are present in ./vendor/bin will then have priority over your identically named system commands (e.g. if you have installed phpunit globally, calling phpunit from a FluidTYPO3 repository root folder will instead use the version we ship with the extension).

This concludes the contribution guide. Feel free to come see us on IRC if you have questions.