Last month I did a major backend refactoring of on of our applications and was feeling happy about how much the code has been simplified and easier to understand. All was well until suddenly, we started getting strange errors in production. Our set of unit and integration tests had passed and I was quite confident that nothing had broken.

With a few hours of debugging and head scratching, I nailed the problem down to some mutable variables that were being shared across multiple threads and sessions. In order to avoid initiating and creating multiple objects of a particular service, we had converted it to a Singleton, and the resulting change caused some variables ending up shared between multiple threads and break down the complete application. We did not have any unit or integration tests developed to test for concurrency and hence could not catch the issues earlier

This experience led me to write test cases for concurrency and subsequent evaluation of technologies appropriate for developing concurrent applications. A few hours of reading opened my eyes to how much the world of concurrent programming for the JVM had changed in the last few years. My experience is core java were from Java 2.0 years and with constructs like Thread, Runnable and Synchronized methods.

Java 5.0 has apparently changed the space completely with introduction of the new util.concurrent package. Here are some of the things are now possible now when using the modern JDK

* Manage your own thread pool using Executor Services
* Use the concept of callable to asynchronously call and receive responses from thread
* Coordinate threads using Countdown latches (similar to Semaphores in C++)
* Better data structures for data exchange between threads - BlockingQueue
* Collection data structures - ConcurrentQueue, ConcurrentHashmap for better performance
* Better granularity in synchronizing blocks of code (rather than using object level 'synchronized' methods) using Locks

Further exploring the topic I came across new libraries that were available to implement concurrency using implementation patterns like Memory Transactions and Actors. The prominent ones for the JVM were Akka and GPars.

Akka is implemented in Scala and supports both of the patterns mentioned - it supports software transactional memory (STM) by using specialized transactional primitives that can be modified only within defined transactions. It also supports the concept of defining Actors - Objects that extended an Actor Object would always have their methods executed in separate threads. The Actor pattern of concurrency is also supported by the Groovy GPars library.

Since Grails obviously played best with Groovy, and also since the implementation of concurrent did not involve complex synchronizations, I chose to implement concurrent Spock tests using GPars. The combination of Spock’s highly expressive specification language and the clean design of GPars makes writing tests quite enjoyable.

For example Spock allows Data Driven testing and allows you to wite tests like

class MathSpec extends Specification {
    def "maximum of two numbers"() {
        expect:
        // exercise math method for a few different inputs
        Math.max(1, 3) == 3
        Math.max(7, 4) == 7
        Math.max(0, 0) == 0
    }
}

GPars allows you to implement concurrent execution with a pool of threads and groovy collections like so:

withPool {
    def selfPortraits = images
        .findAllParallel{it.contains me}
        .collectParallel {it.resize()}
}

Combining these two results is extremely intuitive parallel test cases. I have included one of our example test cases below:

package com.squareprism.xyz

import grails.test.mixin.TestMixin
import grails.test.mixin.support.GrailsUnitTestMixin
import spock.lang.Specification
import static groovyx.gpars.GParsPool.*

/**
 * See the API for {@link grails.test.mixin.support.GrailsUnitTestMixin} for usage instructions
 */
@TestMixin(GrailsUnitTestMixin)
class ParallelXYZGeneratorSpec extends Specification {

    // data array for test data
    def dataArray = [
            [11.26, 75.78, 25, 11, 1973, 4, 55],
            [11.26, 75.78, 25, 11, 1973, 4, 55],
            [11.26, 75.78, 25, 11, 1973, 4, 55],
            [11.26, 75.78, 25, 11, 1973, 4, 55],
            [11.26, 75.78, 25, 11, 1973, 4, 55],
            [11.26, 75.78, 25, 11, 1973, 4, 55],
            [11.26, 75.78, 25, 11, 1973, 4, 55],
            [11.26, 75.78, 25, 11, 1973, 4, 55]
    ]
    def results // collect results here

    def setup() {
    }

    def cleanup() {
    }

    void "test Parallel XYZ Generation"() {
        given:
        withPool(8) { // use 8 parallel threads
            results = dataArray.collectParallel{
                return XYZGenerator.calculate(it[0], it[1], it[2], it[3], it[4], it[5], it[6])
            }
        }


        expect:
            results.size() == 8

    }
}

It could kill your SEO rankings

During the last few weeks we were experimenting with how we can improve our SEO rankings for Jaatakam.

We had some pretty basic stuff in our pages as content as far as SEO keywords were concerned. Nothing targeted, but just enough content to inform the user about the website. We had found some pretty good improvement with impressions and views after we optimized our Terms and Privacy policy pages earlier.

Having had some decent success, we went about adding more content to our page to have a better coverage of keywords (Yes. we wanted to be ranked better for those top 100 keywords!). Added some interesting questions and answers that were relevant to the site. Created a separate FAQ page. Added the page to the Sitemap. Let Google know about it. And waited.

The result. In two weeks our page ranking dropped and the number of impressions dropped by 60%.

On frantic analysis, found that the keyword density for those KWs that were helping us get so many impressions had dropped due to the new content added. And that resulted in far fewer hits than earlier.

Lesson learned. Be careful with adding new text content to your site as it could hurt SEO in a big way. One possible solution could be adding the new content to a separate blog page and convert users from that page to your site.

As of now, we have removed the added page and waiting for results. It may be a couple of weeks before we see any change, but we will update here when we see any improvements.

One Line Advice = ‘Don’t Include in your Single Page’

Single Page Applications written in Javascript are very popular these days. All content loads in a single page and when the user clicks on links they see ‘blazing’ performance since pages don’t load from the server.

Almost all Apps today need to include pages around the Terms and Conditions (TOC) of use and Privacy Policy pages. Often times these pages have content that runs into many paragraphs and could be a significant portion of your site/apps content.

What we don’t realize when we add these monstrous pages of legal ramblings is how they impact our websites SEO in a negative fashion. When we have 100 words or relevant content in our app and 2000 words of legalese all included in a single page, the relevant content becomes 5% of overall site from a search engine perspective. Important keywords will have a much lower density and this has a significant effect on your page rankings.

The solution. Get rid of TOC and Privacy from your single page and add them as separate page URLs. Use your robot.txt to disallow the terms and privacy pages from getting indexed. Soon you will start seeing much more relevant content keywords within your webmaster console and better impressions overall.

See below a sample robots.txt on how you should configure your website. Reach out to your webmaster now.