PHP
Article

Popular Users per Language and Region with Silex and Github

By Parham Doustdar

The Github API V3 is a very powerful set of API endpoints that you can use to retrieve useful information about your repositories, activities, and events. Not only that, it allows you to access public information on others. The fact that it has libraries in many languages and platforms is also a big plus; it allows you to get down to writing code very fast.

The data you can get through the GitHub API V3 is mostly what you can see on the GitHub web interface, except you can use the data in your application to come to interesting conclusions. That is what we are going to do in this article.

Younes Rafie has written an article titled How to Use Github’s API with PHP, in which he introduces a lot of what we are talking about here, although in a different fashion. If you haven’t read that article, though, do not fear! We have you covered.

Github logo

The Concept

As Brandon Savage notes in his article, The definitive guide to contributing to open source:

There are millions of PHP developers, but only a (large) handful of contributors, authors and maintainers of open source software. Packagist reports a little under 50,000 packages in the PHP ecosystem. This is a huge number of open source packages, but is a small number compared to the PHP developers in the world.

Now, what if there was a web service in which we could specify a location and a language (e.g. PHP) and get a sorted list of contributors to open-source? It would certainly make Brandon very happy.

In our web service, we will have one endpoint (i.e. action, URL) that shows the most popular ten creators. The most popular creators are the people in a location who have the largest number of stargazers. For this, we will retrieve each person’s repositories and add up the number of times each one has been starred.

The GitHub Awards project does something similar, although in a much more complicated manner. It uses tools such as the GitHub Archive, which are outside of the scope of this article. Younes Rafie covered that approach somewhat, too, in this post. However, what we’ll do here is simply sort users by their number of followers, since that is a good indication of popularity as well. To view the full source code of this app, check out its Github repo.

Gearing Up

Setting Up the Development Machine

The easiest way to get started is to use either Laravel Homestead, or our very own Homestead Improved. Regardless of which one you use, if you open your homestead.yaml configuration file, you will see the default configuration, which is to have a site called homestead.app which loads your php files from ~/Code.

sites:
    - map: homestead.app
      to: /home/vagrant/Code/Laravel/public

Getting Silex and Setting Up Homestead.yaml

Being a micro-framework, Silex doesn’t have many rules or conventions. However, it helps to read this article to absorb some basics. If you don’t feel like switching between articles, we’ll link to relevant sections of the Silex documentation throughout this tutorial.

As you might know, Silex puts its index.php file in the web directory as per Symfony conventions. So, your webroot will actually be ~/Code/silex/web, rather than ~/Code/Laravel/public. Let’s go into the sites section of homestead.yaml and fix that:

sites:
    - map: homestead.app
      to: /home/vagrant/Code/silex/web

Then, punch in vagrant up to boot up the VM and once it’s done, SSH into the VM with vagrant ssh. Inside, we do the following to initiate a new working Silex instance:

cd Code
composer create-project silex/silex

Once this procedure is complete (might take a while) create a subfolder in the silex folder called web and inside it a file index.php with the contents:

<?php

require_once __DIR__.'/../vendor/autoload.php';

$app = new Silex\Application();

$app->get('/', function(){
    return "Hello world";
});

$app->run();

If you open up homestead.app:8000 in your browser now, you should see “Hello World” (provided homestead.app is in your hosts file, as per instructions).

Congratulations! You are now ready to roll!

Getting Started With GitHub API V3

We’re going to use KNPLabs’ PHP GitHub API library to retrieve data from GitHub through its GitHub API V3. Let’s add it to our composer.json file by executing the command: composer require knplabs/github-api ~1.4.

Let’s turn on the debug mode by setting $app['debug'] to true, and make a request to the GitHub API V3 to give us the details of a user named parhamdoustdar. Update index.php.

<?php
require_once __DIR__.'/../vendor/autoload.php';

use Github\Client;
use Silex\Application;

$app = new Application();
$app['debug'] = true;

$app->get('/', function (Application $app) {
    $client = new Client();
    $parham = $client->user()->show('parhamdoustdar');
    
    return $app->json($parham);
});

$app->run();

If you go to http://homestead.app:8000, you should see parhamdoustdar’s details in json:

{
    "login":"parhamdoustdar",
    "id":352539,
    "avatar_url":"https:\/\/avatars.githubusercontent.com\/u\/352539?v=3",
    "gravatar_id":"","url":"https:\/\/api.github.com\/users\/parhamdoustdar",
    "html_url":"https:\/\/github.com\/parhamdoustdar",
    "followers_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/followers",
    "following_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/following{\/other_user}",
    "gists_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/gists{\/gist_id}",
    "starred_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/starred{\/owner}{\/repo}",
    "subscriptions_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/subscriptions",
    "organizations_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/orgs",
    "repos_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/repos",
    "events_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/events{\/privacy}",
    "received_events_url":"https:\/\/api.github.com\/users\/parhamdoustdar\/received_events",
    "type":"User",
    "site_admin":false,
    "name":"Parham Doustdar",
    "company":"TENA",
    "blog":"http:\/\/www.parhamdoustdar.com",
    "location":"Tehran",
    "email":"parham90@gmail.com",
    "hireable":true,
    "bio":null,
    "public_repos":9,
    "public_gists":2,
    "followers":3,
    "following":0,
    "created_at":"2010-08-03T07:56:17Z",
    "updated_at":"2015-03-01T20:01:26Z"}

If you didn’t understand what exactly our controller is doing, don’t fret! We’ll go through it together.

On line 8, we’re turning on the debug mode as previously mentioned.

Inside the anonymous function that will be called whenever the root route (/) is accessed, we create an instance of Github\Client. Note that we’ve added an argument which is of type Silex\Application. As noted in the usage section of the Silex manual:

The current Application is automatically injected by Silex to the Closure thanks to the type hinting.

In the next line, we’re calling the user() method on the client object to get an object of type Github\Api\User, which is used to get information on users. We then simply call show() on this object, passing in the username we want to get information on.

The last line uses the json() helper method on the Silex\Application object to render our example json response.

The App Directory

You might have noticed that our project doesn’t have a location to put our source code in. Having one index.php file is fine when you are writing a small piece of code to check that everything is working smoothly, but it’s now time for us to create a folder to house our app’s source code.

To do this, create a folder in your silex directory, next to your web folder, and call it App. With that out of the way, let’s edit composer.json and add the autoload information:

{
    ...
    "autoload": {
        "psr-0": { "Silex": "src/" },
        "psr-4": {
            "App\\": "App/"
        }
    },

}

Next, run composer dump-autoload to regenerate your autoload files. We are now ready to get to the real thing!

GitHub Users in a Location

The first thing we need for our toplist is to get a list of usernames in a particular location that have repositories in the specified language. We can use the Search Users API for this.

If you look at that page, you will see that we can specify multiple parameters when searching for users. However, what we’re interested in is location and language, which can be specified in the q parameter. If you have used the GitHub search box before, you will notice that the q parameter is what you’d type into that text field.

Let’s go ahead and create a service that searches for users who live in the location we want, and have repositories in the programming language we need. We’ll call it UserSearcher and put it into App/Services.

<?php
namespace App\Services;

use Github\Client;
class UserSearcher
{
    
    /**
     * @var Client
     */
    protected $client;
    
    public function __construct(Client $client)
    {
        $this->client = $client;
    }
    
    public function retrieveByLocationAndLanguage($location, $language)
    {
        $searchString = "location:${location} language:${language}";
        $results = $this->fetchSearchResults($searchString, 10);
        
        return $this->extractUsernames($results);
    }
    
    protected function fetchSearchResults($searchString, $count)
    {
        $results = $this->client
            ->search()
            ->setPerPage($count)
            ->users($searchString, 'followers')
        ;
        
        return $results['items'];
    }
    
    protected function extractUsernames(array $results)
    {
        $usernames = [];
        
        foreach ($results as $result) {
            $usernames[] = $result['login'];
        }
        
        return $usernames;
    }
    
}

As you can see, the UserSearcher::retrieveByLocationAndLanguage() method first constructs a search string. If you send Tehran and PHP as the first and second arguments to this function respectively, the string will be location:Tehran language:PHP.

It then retrieves the list of top ten search results using Github\Client. As you have probably guessed, it first gets the search API object (Github\Api\Search) from the client, then sets perPage to 10, meaning that it will only request ten search results. Afterwards, it sends the search string to GitHub and returns the results. Note that the results of the search are sorted by the number of followers and in the items index of the returned array.

The method extractUserNames() simply does what you’d expect: it loops through the results and adds each user’s login to an array, then returns it.

What do we have at this point? Ten usernames. Ten usernames that we can query for their repositories, and then add up the number of stargazers for all the repositories under that user’s name.

Ready?

Stargazers of Repositories

Below is the code for our class, App\Services\StarGazerCalculator:

<?php
namespace App\Services;

use Github\Client;
class StarGazerCalculator
{
    
    /**
     * @var Client
     */
    protected $client;
    
    public function __construct(Client $client)
    {
        $this->client = $client;
    }
    
    public function calculateStarGazersByUsername($username)
    {
        $repositories = $this->client->user()->repositories($username);
        $repositories = $this->filterForkedRepositories($repositories);
        
        return $this->addStarGazersFromRepositories($repositories);
    }
    
    protected function filterForkedRepositories(array $repositories)
    {
        return array_filter($repositories, function($repository) {
            return $repository['fork'] === false;
        });
    }
    
    protected function addStarGazersFromRepositories(array $repositories)
    {
        return array_reduce($repositories, function($starGazersSoFar, $repository) {
            return $starGazersSoFar + $repository['stargazers_count'];
        });
    }
    
}

Retrieving Top Creators

Now it’s time to start writing our final class, TopCreatorsRetriever. This class wraps up the functionality we want – it gets the most popular ten usernames using the UserSearcher class, and then loops through those users, using the StarGazerCalculator to get the number of stargazers for each user.

<?php
namespace App\Services;

class TopCreatorsRetriever
{
    
    /**
     * @var UserSearcher
     */
    protected $searcher;
    
    /**
     * @var StarGazerCalculator
     */
    protected $calculator;
    
    public function __construct(UserSearcher $searcher, StarGazerCalculator $calculator)
    {
        $this->searcher = $searcher;
        $this->calculator = $calculator;
    }
    
    public function retrieve($location, $language)
    {
        $usernames = $this->searcher->retrieveByLocationAndLanguage($location, $language);
        $map = $this->createUserStarGazerMap($usernames);
        
        return $this->sortMapByStarGazers($map);
    }
    
    protected function createuserStarGazerMap(array $usernames)
    {
        $results = [];
        
        foreach ($usernames as $username) {
            $results[$username] = $this->calculator->calculateStarGazersByUsername($username);
        }
        
        return $results;
    }
    
    protected function sortMapByStarGazers(array $map)
    {
        arsort($map, SORT_NUMERIC);
        
        return $map;
    }
    
}

The Service Provider

As you may know, in order to add our classes to the dependency injection container provided with Silex, we need to create a service provider and then register it on the application. That means we need to create a service provider in order to use our newly written class TopCreatorsRetriever. We don’t need to register the other two classes though, because those two classes are there for internal use by our high-level class, TopCreatorsRetriever.

Create a new folder under App/ and call it Providers. Then, create a file named TopCreatorsRetrieverServiceProvider.php which will house the code for our class.

<?php
namespace App\Providers;

use Silex\ServiceProviderInterface;
use Silex\Application;
use Github\Client;
use App\Services\UserSearcher;
use App\Services\StarGazerCalculator;
use App\Services\TopCreatorsRetriever;
class TopCreatorsRetrieverServiceProvider implements ServiceProviderInterface
{

    public function register(Application $app)
    {
        $app['topCreatorsRetriever'] = $app->share(function() {
            $client = new Client();
            $searcher = new UserSearcher($client);
            $calculator = new StarGazerCalculator($client);
            
            return new TopCreatorsRetriever($searcher, $calculator);
        });
    }

    public function boot(Application $app)
    {
        
    }
}

Note: Do not remove the empty boot() function. It is required to exist, even if it’s empty, so that our class would conform to the ServiceProviderInterface interface.

Using TopCreatorsRetriever In Our Controller

It’s finally time to see all our work in action. We will define a route in index.php called creators, that gets the location and language as arguments, and then returns a JSON array, with usernames as keys and the number of stargazers as values. Don’t forget that as noted before, we also need to register the service provider we created earlier.

<?php
require_once __DIR__.'/../vendor/autoload.php';

use Silex\Application;
use App\Providers\TopCreatorsRetrieverServiceProvider;

$app = new Application();
$app['debug'] = true;
$app->register(new TopCreatorsRetrieverServiceProvider());

$app->get('/creators/{location}/{language}', function (Application $app, $location, $language) {
    $data = $app['topCreatorsRetriever']->retrieve($location, $language);
    return $app->json($data);
});

$app->run();

Very simple. The only things that have changed are the registration of our service provider, and our route, which contains two lines only:

  1. The first line gets the data from the TopCreatorsRetriever class, and
  2. The second line turns that data into json.

Now, if you send a request to http://homestead.app:8000/creators/Tehran/PHP, you will see all the package creators with the highest number of stargazers who live in Tehran:

{
    "sallar":198,
    "sepehr":86,
    "reshadman":49,
    "moein7tl":49,
    "alijmlzd":13,
    "atkrad":4,
    "AmirHossein":3,
    "Yousha":0,
    "voltan":0,
    "zoli":0
}

Congratulations! You now know who the most popular package creators are in any area!

Conclusion

We now have an application which gathers meaningful data from GitHub through a very basic mechanism: we retrieve the top users in an area who have a repository in the language we want, and add up the stargazers for all of their repositories.

Hopefully, this tutorial has taught you something new and useful. Please leave your feedback in the comments below and don’t forget you can check out the full source code here!

  • sebastiaan hilbers

    Very nice!

  • mhmd

    parham very good ! ‘from care man’

  • http://mustafa.my.id M Zaenal Mustofa

    nice tutorial

  • http://mateusztymek.pl/ Mateusz Tymek

    Cool! Can you post whole app on Github?

    • parhamdoustdar

      Certainly. I’ll have that done soon.

      • http://www.bitfalls.com/ Bruno Skvorc

        Added, see last link in conclusion, thanks!

  • http://www.dev-metal.com/ Chris

    Out of curiosity: Do you have numbers how many PHP developers exist in the world ? Would be awesome to see some stats!

    • parhamdoustdar

      Not really, no. Even these usernames and statistics are for people who own repositories, not for people who code in a particular language. For example, if I code in PHP but don’t have a GitHub repository, it requires a quite different approach to count me in among PHP programmers.

Recommended

Learn Coding Online
Learn Web Development

Start learning web development and design for free with SitePoint Premium!

Get the latest in PHP, once a week, for free.