Krwkrw 0.1.3 Released

Just pushed the latest release (0.1.3) of Krwkrw to Maven central.

Krwkrw is a web scraper. You can read how it came into being here

A quick run down of the changes in 0.1.3.

1. Ability to express URL's to be included/excluded using Regex pattern. For example:
   

   Krwkrw crawler = new Krwkrw(action);
   crawler.match("(\\S+)(/projects/)(\\S+)")
   

   makes sure that only contents in the /projects/ path would be processed while
   

   Krwkrw crawler = new Krwkrw(action);
   crawler.skip("(\\S+)(/projects/)(\\S+)")
   

   will fetch and process all the contents except, the ones in the /projects/ path
   
2. Ability to have random delays in between requests.
   Before now it was only possible to set the seconds to wait between requests. For example:
   

   Krwkrw crawler = new Krwkrw(action);
   crawler.setDelay(5) // waits 5 seconds between requests
   

   With the 0.1.3 release, it is possible to have random delay; that is, the requests will be delayed by number of seconds picked randomly from a lower and upper bound, for example:
   

   Krwkrw crawler = new Krwkrw(action);
   // waiting seconds will be any number between 5 and 20
   crawler.setDelay(5, 20) 
   

3. Change in API. doKrawl method replaced with crawl
   
4. Fix issue where it was possible for the crawler to crawl pages outside the origin url
   
5. Some minor improvements here and there...
   

If using Maven as your build tool, you can add it to your project via:

<dependency>
<groupid>com.blogspot.geekabyte.krwkrw</groupid>
<artifactid>krwler</artifactid>
<version>0.1.3</version>
</dependency>

If using Gradle, then:

dependencies {
compile "com.blogspot.geekabyte.krwkrw:krwler:0.1.3}"
}

The Javadoc can be accessible online here.

You can also check out Krwkrw on Github

What Does Changing Someone Else's Code Have In Common With Defusing A Bomb?

A while back, the above picture bobbled up my twitter stream...it was a morning of a weekday if I remember correctly...and as I was about to bop my head in agreement and push the retweet button: a gesture to show the picture also holds for me...It slowly struck me: that was not the case... this does not hold true for me...maybe it did before, but not anymore.

See, I understood the sentiments expressed in the picture so perfectly well. Either due to building on a codebase written by someone else (aka a former employee), or working on the same codebase with other developers. That apprehension of approaching someone else's code. That feeling of things blowing up, in unpredictable and unexpectable ways was just too familiar.

How Well Do You Know Your Collection API?

I recently spent some time going over the Java collection API and even though, it can be said it is one of the most used API in everyday programming with Java, I still had some "Oh I totally forgot about that" moments to "I did not even know that was so" realizations.

I will note down some of the interesting observations in this post, being as succinct as possible and backing with code examples when needed. It is intended to be a place I can always consult to get an overview of the API.

Embedding ElasticSearch In a Spring Application

This blog post will show how to have ElasticSearch embedded in a Spring Application.

Motivation

Why Embed ElasticSearch in an application? Simple. To reduce the infrastructure overhead an application needs in other to run, making it more portable.

It is also for the same reason you will want to embed, say a web server, within your application.

It is especially appropriate to have ElasticSearch embedded when the use case does not require the distributed features of ElasticSearch and all that is needed is just a single node.

Video Courses? Maybe, Maybe Not.

Sometimes last week I got to learn that one of my blog posts inspired folks at Webucator to make a video tutorial.

The video, which is seen below, is about a feature in Spring MVC. it was inspired by the post Injecting and Binding Objects to Spring MVC Controllers




I recently published a series of post on introducing Servlet based web application development in Java. The original idea was supposed to have the topic covered in a single post, not a series, but as soon as I started writing, it became obvious that the topic at hand is too broad to fit into a single post.

This sort of realization is fast becoming the norm when considering blogging tutorials that relates to Java, as the landscape is quite extensive, having various moving parts that an attempt to write short, relatively beginner level tutorials is almost impossible.

With this video, I might have just been inspired to consider video courses, as those seem to provide the apt medium for covering the vast landscape that is Java software development.

...or maybe not? as making videos definitely requires a whole lot more time... time, the luxury I currently do not have much of. :)

We'll see..

...and oh, by the way the folks at Webucator do offer Java Fundamental courses... You might want to check them out.

Introduction to Spring Boot as a Servlet Based Web Application Framework

This is the fifth and last post in the series on Introduction to servlet based web application in java. The previous posts in this series are:

1. Introduction to the Servlet API, Web Server and Servlet Container
2. Introduction to .war Packaging and Maven
3. Quick Overview of Basic Maven Concepts
4. How to Create .war Files with Maven Archetypes and Deploy to Tomcat
5. How to Create .war Files with IntelliJ and Deploy to Tomcat

These posts have touched on web servers and containers, the servlet API, packaging war files and how to achieve this using built in support in IDE or via maven archetype. This post would conclude the series by taking a look at Spring boot.

How to Create .war Files with IntelliJ and Deploy to Tomcat

This post is the fourth post in the series of post on Introduction to Servlet based web application development in Java.

Previous post shows how to create Maven projects, package and deploy to either a standalone server or an embedded server, by directly using the Maven archetype; but since in practice, an IDE would be used for development, this post shows how to do the same using an IDE.

Intellij IDEA is used in this post. If using other IDEA (say Eclipse or NetBeans) you would have to consult the help section or online documentation for the corresponding steps in the IDE you use.

This post is adapted from Developing a Java EE application. A post which shows how to create and run a JEE web application, but it does not accomplish this with Maven and uses Glassfish as the application server instead of Tomcat which we are interested in.

What this post does is to modify the steps to include usage of Maven and then show steps on how to deploy to Tomcat