MeasureUp 2016: Distributed Systems 101

So guess what. It looks like I have been selected to speak at the second annual MeasureUp conference in Austin TX at the end of July. This year I will be giving a talk on Distributed Systems.

This is a topic that is near and dear to my heart. I recently was submerged (head first) into this new and exciting world and it has re-kindled my passion for building software in a big way.

I am really excited to share what I have learned so far and I am really excited to do it at MeasureUp. There are several Microsoft MVP’s who were selected to speak (Yes, I said MVP’s!) and they still decided to pick little ol’ me. How cool is that?

MeasureUp

When is it
MeasureUp will be happening on Saturday, July 23rd 2016. Breakfast starts at 8:30 and the Keynote by MVP Jeffrey Palermo starts at 9am sharp!

If you are in the Austin area
This is your chance to come hang out with other .NET developers from the community as well as rub elbows some Microsoft MVP’s like Brian Sullivan as well as James Chambers and Simon Timms who have several videos on Channel 9. There will also be great speakers coming in from all over the country (even Canada!) like Kyle Nunnery and Chris Harrington. And if you are into Docker then you wont want to miss the great Gabriel Schenker who regularly contributes to to Los Techies on the subject. And of course me! Woot.

Not from Austin?
Then this is your chance to come visit the The Live Music Capital of the World, eat some great BBQ, and expand your knowledge base with some of the great developers in our industry.

So let’s do this already!
Check out the conference agenda. Docker, ASP.NET Core, continuous deployment, React.JS, Distributed Systems, microsevices, soft skills and more.

Register for the event.

See you in July!

ASP.NET Core RC2 IOptions Configuration: An alternative to Enumeration

As of today the Enumeration nuget package from the good folks over at Headspring does not work with ASP.NET Core RC2 (formerly known as ASP.NET 5).

There is an alternative that while not exactly the same does allow for you to have a data structure that contains the data you need access to through its properties and nested types. The solution takes advantage of the configuration framework that is now part of ASP.NET Core. You can read the official documentation here.

My problem
With the new ASP .NET Identity system roles have become deprecated. The new hotness is to save everything as a claim. My claims (permissions) list was going to be relatively static and not change very often so I didn’t want to store them in a database of some kind and look them up with every request when doing authorization checks.

Create a JSON file
First we need to put our data into some type of format that can be used by the configuration framework. XML, JSON, and INI files are supported. I will be using JSON. Here is what my file looks like:

{
  "Claims": {
    "Admin": {
      "UserManagement": {
        "ViewUsers": {
          "name": "View Users",
          "id": 100
        },
        "CreateUsers": {
          "name": "Create Users",
          "id": 110
        },
        "EditUsers": {
          "name": "Edit Stats",
          "id": 120
        }
      }
    },
    "user": {
      "products": {
        "EditProducts": {
          "name": "Edit Products",
          "id": 200
        },
        "ViewProductReports": {
          "name": "View Product Reports",
          "id": 210
        }
      }
    } 
  }
}

Next save the JSON file into the root of the project:
claimsInRoot

Create a Data Structure
After creating your JSON file you will want to generate a data structure for it.

Now hold on there partner! Not so fast. Before you go off and start building out some classes by hand you should try this nifty little trick:
1. Highlight your data from the JSON file and copy it to the clipboard.
2. Create a new class file and save it wherever you want. I put mine in the models folder.
3. Delete the class declaration and leave the namespace.
4. Set your cursor between the curly braces of the name space and then go to Edit > Paste Special > Paste JSON as classes. And voila!

This is what was generated for me:

namespace IOptionsSample.Models
{

    public class Rootobject
    {
        public Claims Claims { get; set; }
    }

    public class Claims
    {
        public Admin Admin { get; set; }
        public User user { get; set; }
    }

    public class Admin
    {
        public Usermanagement UserManagement { get; set; }
    }

    public class Usermanagement
    {
        public Viewusers ViewUsers { get; set; }
        public Createusers CreateUsers { get; set; }
        public Editusers EditUsers { get; set; }
    }

    public class Viewusers
    {
        public string name { get; set; }
        public int id { get; set; }
    }

    public class Createusers
    {
        public string name { get; set; }
        public int id { get; set; }
    }

    public class Editusers
    {
        public string name { get; set; }
        public int id { get; set; }
    }

    public class User
    {
        public Products products { get; set; }
    }

    public class Products
    {
        public Editproducts EditProducts { get; set; }
        public Viewproductreports ViewProductReports { get; set; }
    }

    public class Editproducts
    {
        public string name { get; set; }
        public int id { get; set; }
    }

    public class Viewproductreports
    {
        public string name { get; set; }
        public int id { get; set; }
    }

}

Setup the IOptions Configuration
Now that we have our data in a JSON file and our data structure to access it lets wire up the project to make use of it. To do that we need to go into the new startup class which is located in the root of the project.

In the constructor add the new claims.json file to the config builder.

var builder = new ConfigurationBuilder()
    .SetBasePath(env.ContentRootPath)
    .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
    .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
    .AddJsonFile("claims.json", false, true);

Next in the ConfigureServices() method add these two lines anywhere ( I put mine at the top of the method):

public void ConfigureServices(IServiceCollection services)
{
    services.AddOptions();
    services.Configure<Claims>(Configuration.GetSection("Claims"));
    
    //Other code.....
}

Get the nuget package
For the mapping to work we will need to add the necessary nuget extensions package. Right click the project and install the package below:

Microsoft.Extensions.Options.ConfigurationExtensions

Now lets use it
To make use of our new claims we use dependency injection! Add it to the constructor of a controller or where ever you want and it will be available for you to use. Just as an example I am going to add it to my HomeController.

public class HomeController : Controller
{
    private readonly Claims _claims;

    public HomeController(IOptions<Claims> claims)
    {
        _claims = claims.Value;
    }

    public IActionResult Index()
    {
        var claimId = _claims.Admin.UserManagement.CreateUsers.id;
        ViewData["CreateUsersID"] = claimId;

        return View();
    }
}

ASP.NET Core has a new lightweight DI framework built in. When we configured our service above we were essentially setting up our IOC mapping so that when we asked for IOptions in the constructor the frame work would know what to return.

Now we have access to the claims list and we can use it when creating policies for authorization.

What is a Service in Service Oriented Architecture (SOA)

Services in SOA need to fit within the 4 tenets of SOA:

  1. Services are autonomous
  2. Services have explicit boundaries
  3. Services share contract & schema, not class or type or databases
  4. Service interaction is controlled by policy

This means that services should not rely on each other. They should be able to operate alone without any assistance from other services or other outside sources. There should be a clear border around a service defining the business logic, data, classes, types, etc. that belong to it. No other service should have a say or influence over the resources of another service. When services do share something it should only be the schemas and contracts necessary for sending communications. It should never share any of its resources. And services should have a strict policy in place for how each service will physically move communications from one service to another.

Communication is important in Service Oriented architecture
For a service to be autonomous and have explicit boundaries we need to pay really close attention to how they communicate with other services. If the logic or data in one service can influence the resources and/or behavior in another service it will undermine the autonomy and boundaries.

So how do they communicate? Never! Absolutely never ever. We want to strive to have no communication between our services. That being said we will never be able to reach absolute zero communication and nor would we want to. Doing so would diminish the usefulness or our system. So when we do need to communicate it should be in a very controlled fashion and indirectly. The best architectural style that can help us to achieve that is the Bus pattern. In short it provides dumb pipes for sending messages in the form of events. Each service can then make use of pub/sub to subscribe to events from other services. This allows for services to remain autonomous and retain their explicit boundaries by avoiding temporal coupling. But we still have to be extremely careful that we are not using the messages to transfer/duplicate data across services.

We achieve tenets 3 and 4 through messaging. The messages in our system become the contracts and schemas we share between our services and then our physical transport and formatting of those messages becomes our policy.

Autonomy and Boundaries are also important in SOA
Udi Dahan, one of the world’s foremost experts on Service-Oriented Architecture, defines services a little further with this definition:

  1. A service is the technical authority for a specific business capability
  2. All data and business rules reside within the service
  3. Nothing is “left over” after identifying services
  4. Everything must be in some service

These rules drive home the idea that a service has a boundary and everything inside of the boundary belongs solely to that service. To be autonomous you can’t have outside influences affecting what you control. And to define what you control you need to have a boundary.

On a bit of a side note…. I often come across comments that associate autonomy with being independently deployable. When we think about how to deploy a service it puts us in the wrong frame of mind. Autonomy is more about encapsulation and being self governing. Mr. Dahan tells us that services are artifacts of the logical view. So when we start talking about services being independently deployable we are no longer in the realm of talking about how we logically organize things (see the 4+1 architectural view model for more information).

Services stretch across systems and processes
It’s very easy to see services as these things that live somewhere in the back-end our systems. The truth however is that the autonomy and boundaries of a service expand beyond processes and into whatever parts of our system they need to be in. So as we said before if a service is autonomous and in control of its own resources then how would you share the data of your service with a front-end HTML/Javascript UI so you could display content to the users? The not-so-obvious conclusion is that the Javascript code I would need for retrieving data from a service would also have to be owned by the service.

So are you saying that Javascript code belongs in the back-end with my service logic and data? No! A service isn’t a back-end thing. It’s a logical view of how we group stuff together. How we deploy the parts of our service and where they actually live are part of the deployment view and the physical view.

We need a special IT/Ops Service
I briefly want to touch upon the fact that we will inevitably need a way for our services to do something that may violate one of our tenets. For example we may need to communicate with third parties outside of our control or services in different systems who have a different policy for communication. To avoid coupling our services directly to these outside systems we make use of the IT/Ops Service. This abstracts away any of the coupling or dirty deeds we have to do. The IT/Ops Service can be useful in other situations as well and it is meant to handle all of the odd jobs that don’t fit in exactly with other services that are aligned with business capabilities.

View of SOA
So without further ado lets take a look at a generic view of services in SOA. I have tried to capture the ideas we have discussed as well as included some we have not. Try to keep in mind this is more of a logical view and that it doesnt fully represent how we might deploy the services.

SOA Logical View

Things that services are not
I don’t want to go to far down the rabbit trail but I thought it would be worth mentioning that Mr. Dahan has a few examples of what services are not and you can learn a little more about them in his talk he gave at the London NDC titled “Finding Service Boundaries – illustrated in healthcare.” Fast forward to about 14:28 to see what they are. In fact I recommend going back and watching the whole video (or at least the first 40 minutes) after reading this blog post to get Mr. Dahans high level on SOA in general.

For more reading on services check out Udi Dahans article, The Known Unknowns of SOA.

SOA. Let’s remove some of the confusion.

Before you can talk about what Service Oriented Architecture (SOA) is, you need to understand the difference between an architecture and an architectural style.

Udi Dahan defines an architectural style as the, “Do’s and the Don’ts. The Rules. Thou shalt do this, thou shalt not do that in your architecture. But it’s not meant to be a complete form of your architecture. It’s a subset.” (Dahan, ADS Course).

The architecture of your project, your application, or your system will and should be comprised of multiple architectural styles.

For example, in the typical n-tier style application you might make use of MVC on the front-end, layers and onion architecture in the back end, etc.

You have probably used many different architectural styles before and didn’t know it. That’s because many of the things we call architectures are really architectural styles. Layered architecture, onion architecture, hexagonal architecture; these are all really styles and not all inclusive architectures unto themselves. Sometimes we aren’t even lucky enough to have the word architecture to identify it; for example, MVC, bus, and pipes and filters.

So I like to think of architectural styles as the ways we can go about implementing something and an architecture as the actual implementation of those styles we chose to work with.

The idea of styles doesn’t mesh well with most people, “[we] prefer cookie-cutters. Just do everything this way and everything will be ok. Unfortunately, we are building systems that are more complex than that so we have to get [deeper] into the details.” (Dahan, ADS Course)

The Do’s and Don’ts of SOA

So back in the day a bloke named Don Box at Microsoft came up with the four tenets of Service Orientation and they are:

1) Services are autonomous.
2) Services have explicit boundaries.
3) Services share contract & schema, not class or type or databases.1
4) Service interaction is controlled by policy.

So there you go. That’s it. That’s the do’s and don’ts of service oriented architecture. It’s really quite simple… almost too simple. And therein lies one of the issues with its adoption. It is so wide open you are left to interpret a lot of things. Like Udi said we as developers really want that cookie cutter recipe or formula to follow. When we start having to spend brain cycles on something other than coding we start to lose interest.

The simplicity of SOA is its greatest strength; it’s also its reason for low adoption and inappropriate implementations. It sucks that we don’t have much to go on as far as instructions on implementing it. Yet it’s not a bunch of rules binding our hands so we have a lot of freedom to build our system the way it needs to be built. One thing that helps to here is to reiterate that SOA is an architectural style. So when we apply SOA we will most definitely need to apply other architectural styles to create a full architecture for our systems and or applications.

What SOA is NOT

Now that we recognize the service oriented architectural style for what it is we can start taking a look at how it’s being misrepresented and remove some of the confusion about what it is.

1) SOA is not web services (building web services != SOA).

First let’s define what a Web Service is:

Any service whose functions can be invoked via a web call. – The Real Justin Self

An HTTP API call that allows one computer to talk to another computer. (Me: does it have to be over HTTP?). Yes, because when we say web we mean HTTP. – Jeffrey Palermo

A web service linguistically sounds like it should be a type of service. But when you think of a web Service as a “thing” and then compare that “thing” to the tenets of SOA it falls short of items 1 and 2. Especially because web services have a strong inclination to be a request/response communication. This means we will more often than not end up with temporal coupling when we communicate to and from a web service. If we boil this down to the nitty gritty we end up with:

What happens in service A effects service B.

When the functionality or resources of one service effect the functionality or resources of another service the boundaries are no longer explicit and the services are no longer autonomous. (Dahan)

2) SOA is not WSDL.

WSDL is more or less an XML file used to describe a web service endpoint. One blog describes SOA as being “generally based upon WSDLs for defining interfaces.”3 I suppose you could make it a part of your architecture but it is not a requirement (see tenet 4) nor is it the defining foundation of service oriented architecture. SOA is actually more likely to have been based on messaging which is the preferred form of communication in SOA. We also shouldn’t forget that the WS in WSDL stands for web service which we already determined wasn’t SOA either.

In reality it’s just plain inappropriate to compare WSDL to SOA. WSDL is an implementation technique and SOA is an architectural style (you know that already though).

3) SOA is not an Enterprise Service Bus.

Sometimes SOA gets mixed up with the notion of an Enterprise Service Bus (ESB). Most ESB’s are actually not a bus. They are really brokers. The broker pattern like SOA is based on messaging. And when you get into the 4th tenet of SOA you start talking about communication implementations and brokers are one of the possible architectural styles you can apply. The original intent of the ESB was to provide a way to integrate several applications together in a way that would allow inter-application communication. In terms of SOA it’s easy to correlate the idea of hooking several different applications together in a way that would allow them to communicate with hooking up several services in a way that would allow them to communicate.

Again we are back to this idea of how to make services talk to each other. This is another driving point of confusion about what SOA is. The next time you find yourself thinking about how services communicate just go back to tenet #1. Communication between services undermines and breaks autonomy. So in SOA we actually want to AVOID all communications between services. That’s right we want none! So if you read a blog about SOA and they start jabbering away about REST API’s, WSDL, SOAP, JSON over HTTP, or ESB’s then something is wrong with the authors understanding of service oriented architecture.

Ok So what is SOA?
Ahh. That is a great question. And the answer is…….you will have to wait until my next blog to find out. This one is already long enough.

References and Notes:

1 The 3rd tenet of SOA was modified by Udi Dahan to include “or databases” explicitly. Previously it was implied implicitly through tenets 1 and 2 but its importance is often overlooked.

2 Dahan, Udi – Advanced Distributed Systems Course (ADS), http://particular.net/adsd

3 Pool, Kevin – The Great Debate: Microservices vs SOA, http://www.tibco.com/blog/2015/07/24/the-great-debate-microservices-vs-soa/

JIRA edit existing query filter

Recently while trying to make changes to our Scrum Board in JIRA I had some trouble when trying to update the query filter it uses.

Every JIRA scrum board you create has one main query it sits on top of. This query is what populates your board with the tasks you want to manage in your backlog, current sprints, etc.

The problem is that when you get to the page where you can actually edit the JQL query all you get is a “Save As” button. This is very frustrating.

What you need to do at this point is actually USE the filter. Click enter or use the search button (magnifying glass icon) at the end of the bar. After you actually do a search with your new query the button changes from “Save As” to just “Save.” If you don’t actually DO the search with your new query then you won’t be able to save it.

The User Object Test

Have you ever heard of code smells? Basically you can think of them as big red warning flags indicating deeper level issues or concerns with your code that aren’t always apparent at first sight.

I have never really had the good fortune as a professional developer to start work on a project that had had a good, clean, solid code base. Over the years after working on several projects like this I started to notice a trend that revolved around the data models that I like to call data model pollution. What I saw was that the data models contained data that was related but not essential or pertinent to the core of what that model was originally intended to be.

The easiest way to check this is to simply go take a look at your projects User Class. Most applications have users but if your project doesn’t then well I dunno. Try looking at some other class that was one of the first created or designed for you project. Snoop Lion formerly known as Snoop Dog might refer to one of these classes as an OG class.
Now that you have you user class open take a look at the properties and see if there is anything in there that’s out of place or odd. For example consider the properties of this interface for a user class:


interface IUser
{
string AccountNumber { get; set; }
Company Company { get; set; }
int CompanyId { get; set; }
string DepartmentName { get; set; }
bool DisplayWalkthrough { get; set; }
bool DisplayWalkthroughModal { get; set; }
string Email { get; set; }
string FirstName { get; set; }
string LastName { get; set; }
string FullName { get; set; }
ICollection FollowingMe { get; set; }
string JobTitle { get; set; }
DateTime LastLoginDate { get; set; }
string LocationName { get; set; }
string ManagerName { get; set; }
ICollection Notifications { get; set; }
string Picture { get; set; }
bool ReceiveCommentEmails { get; set; }
bool ReceiveCommentNotifications { get; set; }
bool ReceivePointsEmails { get; set; }
string Role { get; set; }
DateTime? StartDate { get; set; }
int Status { get; set; }
string UserName { get; set; }
User.UserStatusValue UserStatus { get; set; }
}

Looks a little funky right? I might consider FirstName, LastName, and JobTitle to be properties of a typical user object but what about DisplayWalkthrough, ReceivePointsEmails, and FollowingMe? Those extra properties have some relation to what a User should be but I would strongly argue that they are not pertinent to the core of what a User should be for the application this code belonged to.

I think over any period of time data model pollution is a natural occurrence that just happens as a side effect of fast feature delivery, code maintenance, and the natural evolution process of an application. As a project grows in size in complexity it becomes an enormous effort to avoid and prevent data model pollution from occurring. It just becomes down right impossible.

I believe the User Object Test is one indicator of something deeper and fundamentally wrong with all of the current enterprise application architecture patterns that exist today. I will delve deeper into this in my next blog.

SEO tip for canonical urls

Search engines see different domains as different sites. For instance:
www.attorneyproz.com” is different from “attorneyproz.com.”

The www part of the url (referred to as the subdomain) is important when considering how to structure your URL’s. So its important to make all of you URLS redirect to one place. Below is a chunk of code you can add to your .htaccess file in your root directory to clean up this multi-domain issue known as Canonicalization.

RewriteEngine On
RewriteCond %{HTTP_HOST} ^socialpatterns\.com$ [NC]
RewriteRule ^(.*)$ http://www.socialpatterns.com/$1 [R=301,L]

To learn more about what this code means and what it does checkout SEO marketing tips from www.socialpatterns.com.