Planning your SCM strategy

Planning your SCM strategy

  December 18, 2018

We have been frequently asked by companies that are planning or are developing a software product about the common issues found trying to achieve a good production workflow, good teamwork and the security of properly handling the code base.
During our years of expertise we developed a strong vision about the most important tools that you need to use if you want to build software in a controlled way.
These tools encourage the implementation of a Software Configuration Management (SCM) strategy within your organization. In turn, this will have a great positive impact in many aspects, some of them technical, but also in many functional and workflow oriented ones.
Today most of software engineers take these tools for granted. But there are cases where companies with small development teams or organizational debts need some help figuring out these stuff. In that spirit we are writing an overview of this topic.
Most of the important problems that could come up are solved by properly installing, configuring and using this tools.
One key fact is that these tools are important regardless of the language or platform that you're using.

First step: Secure your code

You have invested a lot of resources in building your product and you have a few customers using it. The source code is the most important part of that product, so the first issue you have to consider is how to store your code in a secure manner while helping improve the way your team work together without conflicts.

The solution to this is using a Version Control Software. In the beginning there were many different options to choose from, but these days almost everyone is using Git. It's one of the most advanced tools for handling code for large teams and it's easy to find candidates with experience using it.

Of course there are others: SubVersion, Mercurial, Team Foundation Server, Bazaar, etc.

Second step: What about the packages?

It's almost impossible to find an organization that builds software without a VCS, but the same cannot be said about handling the binaries or artifacts that you create to distribute your software. Most engineers don't even know there are tools that can be used for this. And why should they consider using such a tool?

In order to answer this, first we need to think about what are the steps involved in writing software:

1. We begin by writing the code. You can use a minimal tool for doing this, like a simple notepad application, or you could use a heavy IDE with modern code generation techniques, but you almost always end up in tons of text files.
2. Then, depending on your platform you might need to compile the code into a binary or intermediate format. That is a process that takes a source code as an input, and produces binary files as the output. This step might not be required if you're using an interpreted language as Javascript or PHP. But even in those cases, there are usually a transpilation process that creates new forms of code optimized to run in different browsers, and environments.
3. Next you test the application in your local environment.
4. If everything is ok, and you're ready for the production deployment phase. This involves taking the artifact that you tested locally or in a test environment and placing it in the server that is going to expose your application.

To be completely sure that there are no problems with the generated artifact, the best is to always perform the build in a clean environment (we'll explain this later), and in order to avoid repeating this process unnecessarily, the best is to store it in a safe place. This is when the binary repository shows its strength. This tool will store the artifacts in an organized and reliable way so you can download them when you need to deploy, but also when you need to build another software based on a library, for example.

There are many options available: Nexus, Artifactory, Archiva, etc.

Final step: Continuous Integration

Having the source code and the produced artifacts in place is only the beginning. Automating the construction of the applications and libraries will give you many advantages:

• Having a clean and centralized place where the software is going to be built. Usually in many companies, the senior developer or the technical leader is the one that builds the artifacts that are going to production. But what if he or she forgets some work in progress or experiment, and then some unwanted behavior is accidentally introduced in the production ready artifact?
• The building pipeline can be triggered upon changes on the code. This can help in finding possible issues earlier, thus reducing time spent following up on them when found in production, and of course it finally produces better quality deliverables. 
• The building recipes can be standardized, so no wheels are going to be reinvented in that matter.
• You can control who triggers builds and deployments, and have logs about those actions. That at the end means better control of the changes that have some impact in important environments, such as staging, testing, etc.

There are many alternatives for this kind of software, but one of the most well known is Jenkins CI. This open source project stands out due to its very large and active community, a vast list of plugins and extensions, addressing almost all of the problems that can arise when configuring this important part of the SCM strategy.

But there are other good choices as well: TeamCity, Travis, Bamboo, etc.

Bonus track: Versioning scheme

But that is not all, you also need to define how to version your software and libraries, how to handle the release of your artifacts, how to manage dependencies, etc.

For that there are many tools, depending on the platform. For Java developers, Maven is almost a de facto standard, but you can also count with Gradle and Ivy. For .NET platform, you have NuGet, and for Javascript world, you have NPM, Bower and Yarn. All of them can be configured to be used with the binaries repositories, and the CI servers available.

Given the difficulty that can emerge when defining this process in your company, the best is to have the advice of experts.

We have years of experience in this subject, and helped many customers in defining and implementing their SCM strategy, so feel free to contact us and get more information.

See you all in the next post.

Developing modern web-apps with Vaadin @ UNL

Developing modern web-apps with Vaadin @ UNL

  September 24, 2018

Last Wednesday we gave a presentation in Universidad Nacional del Litoral, where we explained some concepts related to developing web applications using the latest version of the Finnish platform.

We started explaining what it means to develop a client-server application nowadays, and then some key features of the platform, like the lack of need of browser plugins and the fact that it is completely open-source.

Then we introduced Web Components, a key technology, that is the ground base for the good looking components that are built-in in the platform. Also some introduction for Polymer Project. For more information about these technologies, we recommend reading about it in the WebComponents official site, the Polymer Project official site, and also at Vaadin's site.

Then we started talking about Flow, the Java API for creating modern web-apps using Vaadin's components in the client side. We showed some small examples covering the most significant features like server side DOM manipulation, routing, data binding and synchronization.

If you want to learn more about Flow, the official documentation site is a good place to start.

Students at Universidad Nacional del Litoral

Finally we developed a small application using Flow, showing how easy is to create a good looking web application that can edit data from a back end using Grid, one of the most advanced components.

After the speech, the students were interested in knowing more about the framework and some real life examples. So we decided to upload to GitHub the application that we developed in front of the audience. We added some tags that show each step in the process, feel free to download the sources and try modifying it to cover other use cases!

Lot of questions!

We also recommended another article that we wrote, telling the main steps for developing a complete application that covers how to invoke external REST APIs from the server side.

The experience was great and we found a lot of interest in the framework, so probably we are going to repeat the presentation again soon in another place.

Keep in touch with us for updates!

Javier explaining Flow basics

Vaadin 10 + Spring Demo application: World Cup Rusia 2018 stats

Vaadin 10 + Spring Demo application: World Cup Rusia 2018 stats

  July 03, 2018

To test out some new interesting web technologies, we decided to go ahead and build a small demo application that will point out ideas that could potentially help a developer when trying to decide which ones to use in a new project. We decided to create a small application with responsiveness in mind, that would show real time and aggregated data about the current Football World Cup event that is taking place in Russia.

Technologies

The chosen technologies where:

• Vaadin 10: The brand new version of the popular framework, that is powered by the new standards being pushed by the most popular browsers.
• SpringFramework: The almost de-facto standard dependency injection framework, that stands as the backbone of almost every java project that are currently being built.
• REST API: Our project uses the API available here. That API exposes data as a REST API with information being delivered in JSON format.

Features

This is a short list of currently supported features:

• The application is responsive, it will correctly display information regardless the device being used. This feature is achieved by using Vaadin & Polymer technologies.
• Depending on the device that you're browsing the site (especially mobile phones), you can add the application to the home screen, so it will look like a native application. The only catch is that it will not function in offline mode (yet, maybe sooner).
• You can follow the matches in realtime. Using WebSockets, the application maintains an opened communication pipe between the browser and the server, so the updates can be informed to the client as soon as they are received from the API

Construction

You need to have a good starting point to start any new project. For us it was the Vaadin Base Project with String starter. By choosing this option, we could benefit of the fact that it already has a tested and working Spring configuration, so it was an easy decision. 

Responsiveness

We achieved this goal, by starting also a parallel development: a new Addon for the Vaadin platform that helps you to build new applications using a simple skeleton that takes care of building the usual visual elements present in a responsive application: hamburger menu, paper-cards and more.

Besides that, we also used some CSS techniques as media queries to make the final touches.

PWA

To allow the application to be added to the home screen of your device, and then showing it as a native application, we generated a PWA manifest, with some properties that you can easily mimic, by using a tool like the App Manifest Generator.

Realtime updates

This feature is really easy to implement, given Vaadin's @Push tecnology, based on websockets. Just adding a simple annotation to your main layout will give life to your bubbling screen.

You can learn more about this technology in Vaadin's official documentation.

Roadmap

These are the things we would like to implement in the near future:

• Support for offline message that tells you that you need connectivity to use the app
• Support for notifications
• Localization

You can reach the demo online, by accessing: http://worldcup.flowingcode.com/ ... Try adding it to the home screen of your mobile device!

If you want to give us feedback, feel free to create issues in the GithHub page.

Finally some interesting links regarding these technologies:

• Documentation about Vaadin 10 Flow
• How to consume REST services with Spring Boot
• Creating a PWA Application
• Server push configuration

So that's it, enjoy the app, and if you want share your thoughts about it!

ComboBox lazy loading with REST API in Vaadin 8

ComboBox lazy loading with REST API in Vaadin 8

  April 09, 2018

In this post we'll explore how to access a REST service in order to display items in a ComboBox in Vaadin 8.

First, we'll implement a REST service with Spring Boot, as described here

The web service will be exposed at http://127.0.0.1:8081/api/countries. It will receive no parameters and return an array containing the name of all the countries known by the JVM (the Geography knowledge of the JVM is amazing!).

@RestController
public class CountriesController {

    @RequestMapping("/api/countries")
    public String[] getCountries() {
        return countries().toArray(String[]::new);
    }    

    private static Stream<String> getCountries() {
        return Stream.of(Locale.getISOCountries())
            .map(countryCode -> new Locale("", countryCode))
            .map(locale->locale.getDisplayCountry(Locale.ENGLISH))
            .sorted();
    }
}

Next, we'll implement a REST client that consumes our REST API

public class CountriesClient {        

    private static String ALL_COUNTRIES_URI = "http://127.0.0.1:8081/api/countries";

    public Stream<String> getAllCountries() {
        RestTemplate restTemplate = new RestTemplate();
        String[] countries = restTemplate.getForObject(ALL_COUNTRIES_URI, String[].class);
        return Stream.of(countries);
    }
}

We'll need to add the dependencies for spring-web and jackson-databind in the project's POM (for a detailed explanation of consuming REST services with Spring, you can check this tutorial https://spring.io/guides/gs/consuming-rest/)

    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-web</artifactId>
        <version>5.0.2.RELEASE</version>
    </dependency>

    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.9.4</version>
    </dependency>

Now, we'll write a small application that loads the countries form the REST client, and displays them in a combobox:

    @Theme(ValoTheme.THEME_NAME)
    public class MyUI extends UI {

        private CountriesClient client = new CountriesClient();

        @Override
        protected void init(VaadinRequest vaadinRequest) {
            VerticalLayout layout = new VerticalLayout();
            ComboBox<String> cbCountry = new ComboBox<>();
            cbCountry.setWidth(300, Unit.PIXELS);
            initializeItems(cbCountry);

            layout.addComponents(cbCountry);
            setContent(layout);
        }

        private void initializeItems(ComboBox<String> cbCountry) {
            cbCountry.setItems(client.getAllCountries());
        }

        @WebServlet(urlPatterns = "/simple", asyncSupported = true)
        @VaadinServletConfiguration(ui = MyUI.class, productionMode = false)
        public static class MyUIServlet extends VaadinServlet {
        }

    }

While this approach works, there is a drawback: all the items are loaded when the UI initializes, and setItems instantiates a ListDataProvider, which stores all the items in memory until the combobox itself is disposed. (This is not so critical in this case, since there are only 250 countries, but imagine a combobox loaded with thousands of items...)

Fortunately, we can do it better by specifying an smarter DataProvider that will take the responsibility of fetching items on demand. For this we'll use the method setDataProvider of ComboBox, that instead of a stream/collection/array of items, receives as parameter a DataProvider<T, String>.

DataProvider<T,F> is an interface where the type parameter T refers to the data type (in this case, the ComboBox<T> item type), and the second parameter is the filter type (which for ComboBox is always String, since the ComboBox is filtered by the text the user writes inside). The Vaadin framework supplies an implementation of DataProvider, named CallbackDataProvider, which has the following constructor:

public CallbackDataProvider(
        FetchCallback<T, F> fetchCallback,
        CountCallback<T, F> countCallback)

The fetch callback returns a stream with the items matching a Query, and the count callback returns the number of items matching a Query (a Query is another object in the Vaadin framework, that contains information about index, limits, sorting and filtering):

    @FunctionalInterface
    public interface FetchCallback<T, F> extends Serializable {
        public Stream<T> fetch(Query<T, F> query);
    }

    @FunctionalInterface
    public interface CountCallback<T, F> extends Serializable {
        public int count(Query<T, F> query);
    }

    public class Query<T, F> implements Serializable {
        private final int offset;
        private final int limit;
        private final List<QuerySortOrder> sortOrders;
        private final Comparator<T> inMemorySorting;
        private final F filter;
        //...
    }

In order to take advantage from the CallbackDataProvider, we'll need to introduce a couple of changes to our REST service (CountriesController).

First, we'll need a method that returns a subset of <count> items, matching a given <filter> and starting at a given <offset>.

    @RequestMapping("/api/countries/list")
    public String[] getCountries(
            @RequestParam(value="filter") String filter,
            @RequestParam(value="offset") int offset, 
            @RequestParam(value="limit")  int count) {
        return countries().filter(country->filter(country,filter))
            .skip(offset).limit(count).toArray(String[]::new);
    }       

    private boolean filter(String country, String filter) {
        return filter.isEmpty() || country.toLowerCase().contains(filter.toLowerCase());
    }

Then, we'll need a method that returns the count of items matching a given <filter>

    @RequestMapping("/api/countries/count")
    public int getCountries(
            @RequestParam(value="filter") String filter) {
        return (int) countries().filter(country->filter(country,filter)).count();
    }

Now, we proceed to adapt the REST client (CountriesClient) to these changes, by adding the following methods:

    private static String GET_COUNTRIES_URI = "http://127.0.0.1:8081/api/countries/list?offset={1}&limit={2}&filter={3}";

    private static String COUNT_URI = "http://127.0.0.1:8081/api/countries/count?filter={1}";

    public Stream<String> getCountries(int offset, int limit, String filter) {
        RestTemplate restTemplate = new RestTemplate();
        String[] countries = restTemplate.getForObject(GET_COUNTRIES_URI, String[].class, offset, limit, filter);
        return Stream.of(countries);
    }

    public int getCount(String filter) {
        RestTemplate restTemplate = new RestTemplate();
        Integer count = restTemplate.getForObject(COUNT_URI, Integer.class, filter);
        return count;
    }

Finally, we integrate the modified service in the UI code:

private void initializeItems(ComboBox<String> cbCountry) {
    cbCountry.setDataProvider(new CallbackDataProvider<>(
        query-> 
            client.getCountries(query.getOffset(),query.getLimit(), getFilter(query)),
        query->
            (int) client.getCount(getFilter(query))
        ));
    }

    private String getFilter(Query<?,String> query) {
        return ((String)query.getFilter().orElse(null));
    }

As a final note, there are other components that also use DataProvider, such as Grid and TwinColSelect.

You can download and run the complete code of this example from github.