Saturday, 11 November 2017

FakeDb, Layout tests and Sitecore 9

Following the previous post on this blog involving an issue we ran into upgrading a Sitecore project from 8.2 to 9, this one is also on the same subject. The issue this time was some unit tests that make use of the FakeDb library began to fail following the upgrade.

Noticing that it was just tests that involved checking the status of layout related fields, with the help of some de-compilation, it was clear that a change had occurred in the Sitecore.Data.Fields.LayoutField.GetFieldValue method that now involve the use of a pipeline components.

I found it was possible to resolve this by adding the following to the app.config file, within the &lgt;sitecore&ggt; element, for the test projects containing the failing tests:
    <pipelines>
      <getLayoutSourceFields>
        <processor type="Sitecore.Pipelines.GetLayoutSourceFields.GetFinalLayoutField, Sitecore.Kernel" />
        <processor type="Sitecore.Pipelines.GetLayoutSourceFields.GetLayoutField, Sitecore.Kernel" />
      </getLayoutSourceFields>
    </pipelines>

Have also submitted a PR to have this added to the default configuration for FakeDb, which, if accepted and a new release created, may remove the need for this amend.

Friday, 10 November 2017

Unit testing HttpRequestArgs with Sitecore 9

Working on a project currently with Sitecore CMS, which is currently undergoing an upgrade to version 9.

We had this piece of code, used in the context of unit testing classes that inherit from HttpRequestProcessor, such as custom page resolvers. To use these classes you override a method Process() that takes an instance of HttpRequestArgs.

Constructing these classes from a unit test isn't straightforward and up to now had relied on a method detailed by James-West Sadler on his blog here.

    public static HttpRequestArgs CreateHttpRequestArgs(string url)
    {
        var httpRequest = new HttpRequest(string.Empty, url, string.Empty);
        var httpResponse = new HttpResponse(new StringWriter());
        var httpContext = new HttpContext(httpRequest, httpResponse);

        var requestArgs = new HttpRequestArgs(new HttpContextWrapper(httpContext), HttpRequestType.End);
        
        // Reflection used to call the private Initialize method on the HttpRequestArgs object, which amongst other things 
        // populates the URL on the object itself.
        var method = requestArgs.GetType().GetMethod("Initialize", BindingFlags.NonPublic | BindingFlags.Instance);
        method?.Invoke(requestArgs, null);

        return requestArgs;
    }

Following the upgrade to Sitecore 9 though this began to fail with a null reference exception in the invoke of the Initialize method using reflection.

Obviously using reflection like this is a little risky, as we have no means of being sure something is going to be backwardly compatible here following an upgrade. There's a feedback item to make these more easily testable here.

To resolve I first used dotPeek to de-compile the assembly and code see the following changes between this method in version 8:

    protected internal override void Initialize()
    {
        this._url = RequestUrl.Parse(this._context.Request);
        this._url.QueryString = this._url.QueryString;
    }

As compared to version 9:

    protected internal override void Initialize()
    {
        this.Url = Sitecore.Web.RequestUrl.Parse(this.HttpContext.ApplicationInstance.Context.Request);
        this.Url.QueryString = this.Url.QueryString;
    }

So my first attempt was to try to pass in the ApplicationInstance object which was causing the null reference exception, like this:

    public static HttpRequestArgs CreateHttpRequestArgs(string url)
    {
        var httpRequest = new HttpRequest(string.Empty, url, string.Empty);
        var httpResponse = new HttpResponse(new StringWriter());
        var httpContext = new HttpContext(httpRequest, httpResponse)
            {
                ApplicationInstance = new HttpApplication()
            };

        var requestArgs = new HttpRequestArgs(new HttpContextWrapper(httpContext), HttpRequestType.End);
        
        // Reflection used to call the private Initialize method on the HttpRequestArgs object, which amongst other things 
        // populates the URL on the object itself.
        var method = requestArgs.GetType().GetMethod("Initialize", BindingFlags.NonPublic | BindingFlags.Instance);
        method?.Invoke(requestArgs, null);

        return requestArgs;
    }

This didn't work though, as still the Context property was null.

Next was an attempt to mock this property, as follows:

    public static HttpRequestArgs CreateHttpRequestArgs(string url)
    {
        var httpRequest = new HttpRequest(string.Empty, url, string.Empty);
        var httpResponse = new HttpResponse(new StringWriter());
        var httpContext = new HttpContext(httpRequest, httpResponse);

        var mockApplication = new Mock<HttpApplication>();
        mockApplication.SetupGet(x => x.Context).Returns(httpContext);
        httpContext.ApplicationInstance = mockApplication.Object;

        var requestArgs = new HttpRequestArgs(new HttpContextWrapper(httpContext), HttpRequestType.End);
        
        // Reflection used to call the private Initialize method on the HttpRequestArgs object, which amongst other things 
        // populates the URL on the object itself.
        var method = requestArgs.GetType().GetMethod("Initialize", BindingFlags.NonPublic | BindingFlags.Instance);
        method?.Invoke(requestArgs, null);

        return requestArgs;
    }

Unfortunately though Context is not a virtual property, and as such can't be mocked by tools like Moq.

Final, and successful, attempt was to revert once again to reflection - possibly pushing another problem down the line, but looking a reasonable solution for now.

    public static HttpRequestArgs CreateHttpRequestArgs(string url)
    {
        var httpRequest = new HttpRequest(string.Empty, url, string.Empty);
        var httpResponse = new HttpResponse(new StringWriter());
        var httpContext = new HttpContext(httpRequest, httpResponse);

        var requestArgs = new HttpRequestArgs(new HttpContextWrapper(httpContext), HttpRequestType.End);
        
        // Reflection used to set the protected Url property
        var property = requestArgs.GetType().GetProperty("Url", BindingFlags.Public | BindingFlags.Instance);
        property?.SetValue(requestArgs, new RequestUrl(httpRequest));

        return requestArgs;
    }


Wednesday, 16 August 2017

Native, responsive, AMP and PWA  - options for mobile

Cross-posting an article written for my company blog, looking into different options and technologies available for building web applications suitable for mobile - touching on Xamarin, Cordova, PWAs, AMP and architectural patterns.

Wednesday, 24 May 2017

Let's Encrypt on Azure Web Apps - Key Expiration Issue

I run a side-project website on Azure PaaS, for a music library business called The Perfect Music Library. For a year or so this has had an SSL certificate provided by Let's Encrypt.

The certificates provided for this service expiry relatively quickly, and so in order to keep this updated I've set up and deployed a site extension and web job using the excellent instructions provided by Nik Molnar.

Recently though I noticed that the most recent update had failed which meant browser warnings of an outdated certificate were being seen by visitors that attempted to access the site. It turns out the issue was a key that's created as a process of setting up a "service principal" (step 5 of instructions linked above) had expired after a year or so.

To diagnose this I opened up the Azure portal, located the web application in question and navigated to the list of web jobs. Within that, right-clicking on the letsencrypt.siteextension.job allows the view of the associated logs. Looking at a failed job named Functions.RenewCertificate revealed this error: Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.RenewCertificate ---> Microsoft.IdentityModel.Clients.ActiveDirectory.AdalServiceException: AADSTS70002: Error validating credentials. AADSTS50012: Invalid client secret is provided.

Some googling of that error took me here that revealed the problem. To fix I searched for "App registrations" in the portal, selected the AD application created as part of the initial setup instructions and navigated to the Keys blade. Sure enough, the single key had expired a couple of months ago.

I created a new key and set it to Never expire (only recently an option I understand) and took a copy of the value generated.

With that I went back to the web app in the portal, located Application settings and updated the value of letsencrypt:ClientSecret with the new value.

When the job next ran, it did so successfully so am back with a secure certificate on the website.

Monday, 8 May 2017

Gotcha with Updating .Net Core Using VS.Net 2017

A little while back I put together a very basic ASP.Net Core application and blogged about my first impressions in transitioning from "classic" ASP.Net - can we call it that now?! - here.

Just today thought would be a good idea to update it to the latest version, particular the change in the format of the project file from JSON (project.json) to XML (.csproj). With VS.Net 2017 this was very straightforward, but I found one gotcha that may or may not affect others, so thought would record it here.

When opening the project - originally created in VS.Net 2015 - in VS.Net 2017 a message comes up saying a one-way update needs to happen. I let that complete and it ran successfully. Unfortunately after that VS.Net froze. Closing and restarting the application a few times didn't seem to help - after a few seconds the IDE became unusable and had to be stopped using Task Manager.

The issue seemed to be with a hidden .vs folder that VS.Net creates, containing various developer/machine specific settings. Deleting that - it gets recreated automatically - resolves the problem.

Monday, 1 May 2017

Extending Azure AD B2C with Custom Application Permissions

I recently had chance to work on a proof of concept using Azure AD B2C, which didn't in the end progress but was worth a write-up in case others are working on similar projects, or of course it's something I need to refer back to in future.

Azure AD B2C provides cloud based identity and access management with a consumer focus, supporting social media logins and allowing for controlled access to business to consumer applications.

The particular challenge I looked at followed a recognition that a given client may have several applications that require authentication and authorisation services, but each of them will have their own specific detail around these aspects. Azure AD B2C offers a model of user accounts and groups, however this generic structure may not provide sufficient detail and flexibility for all applications that want to use it.

The proof of concept worked up then was to provide features to extend Azure AD B2C, by building an API that applications can call into, once the user is authenticated, to retrieve additional permission details that are specific for that application.

The following diagram shows the various components involved in this proof of concept, that will be discussed further in this article:

Configuring Azure AD B2C

The first step was to create an instance of Azure AD B2C itself via the Azure portal. One thing that confused me for a little while was that having created it, it doesn't appear as a resource in the same way that most other things like web apps, SQL databases etc. do. The reason for this is that when working with Azure you need to be in the context of a directory, and effectively you are creating a new one in this operation. So it's necessary to switch to the directory using the menu available in the top right:

Social media accounts

With the directory created various related features can be configured. Firstly social media logins. This is a useful means of providing authentication services to a web application that allows users to re-use their existing social media accounts such those with Google and Facebook. There's no need for them to create a new password, or for you, or Azure AD, to store it.

In the screen shot below I've configured the directory to allow for local account login, along with Facebook social media login.

The configuration requires an associated Facebook account to be set up and the client Id and secret from that recorded within Azure. Importantly the Facebook application itself needs to be configured to allow for incoming requests from Azure, which is provided within the Facebook Login product, under Valid OAuth redirect URIs:

More detail on this can be found at this walk-through article.

Defining profiles

Once authentication methods are defined, the details of the particular policies applied when users register, login and carry out other operations with the directory can be set up. These will include information such as the specific fields requested when signing up, as well as the details provided back to the web applications using the directory as claims.

Registering an application
One or more applications can be registered with the directory and assigned to the profiles created in the previous step. They key thing to configure here is the reply URL, which is the endpoint where Azure AD B2C will return any tokens that the application requests. An application identity and secret will be generated that will be required for the applications authenticating with the directory.
Managing user and groups

Of course user account and group assigments can be managed via the portal as well, but it's likely that won't be the primary means of managing this data. In this proof of concept I've looked to do so via the web applications, which I'll come to shortly.

Application specific permissions

SQL Database

In order to provide scope for application specific permissions the first step in this proof of concept was to create a database to hold this additional detail, the structure of which is shown below:

It's quite straightforward, consisting of a mapping between registered users and applications (which users have access to which applications), and, within that, what permissions ("read", "create" etc.) are held by each user on each application. Clearly there's a lot of simplification in place here - were the solution to be extended to something real-world, it would likely require a more structured definition of "permissions" that simple text strings, and we'd likely want to hold these permissions in relation to the user group instead of as well as the user account itself.

Permissions administration web application

I then built a simple ASP.Net MVC application that could be used to manage the data in the database. There's no real need to show the code for this as it's just a simple "forms over data" web application. But the idea was to show that once a user has registered with the directory, an administrator can provide additional permissions for them for the appropriate applications they have access to, once their registration has been moderated according to business rules.

Permissions API

The next component was a an ASP.Net Web API project used to provide two functions as REST operations. Firstly a GET to retrieve the set of permissions for a given user and application. And secondly a POST that records a registration of a new user account with some default, likely quite restrictive, permissions.

The API was hosted behind another Azure component, an API Gateway, which can be used to provided various general purpose API related functions, such as caching, throttling and rate-limiting. Many of these features have most value with an API that's being released for public use, so that subscribers can be managed and potentially monetised, but it also has some value within for a more internal focused API, particularly around authentication.

Effectively the sample we application is a client of the API, so it's registered as a subscriber, which exposes a key. This key is then provided in a header when the sample app calls the API, using the URL of the management service rather than the API directly.

The following code shows the two Web API controller action methods, along with the authentication check that the request provides the necessary header key and value.

    public class ApplicationsController : ApiController
    {
        private readonly AppDbContext _db = new AppDbContext();

        [Route("api/applications/{applicationName}/users/{userId}/permissions")]
        [SwaggerOperation("GetPermissions")]
        public async Task<HttpResponseMessage> GetPermissions(string applicationName, Guid userId)
        {
            if (!IsAuthorized())
            {
                return new HttpResponseMessage(HttpStatusCode.Unauthorized);
            }

            var permissions = await _db.UserAppPermissions
                .Where(x => x.Application.Name == applicationName && x.User.Id == userId)
                .Select(x => x.Permission)
                .ToListAsync();
            return Request.CreateResponse(HttpStatusCode.OK, permissions);
        }

        [Route("api/applications/{applicationName}/users")]
        [SwaggerOperation("AssociateUserWithApplication")]
        [SwaggerResponse(HttpStatusCode.Created)]
        [SwaggerResponse(HttpStatusCode.NoContent)]
        [SwaggerResponse(HttpStatusCode.NotFound)]
        public async Task<HttpResponseMessage> AssociateUserWithApplication(string applicationName, [FromBody]AssociateUserWithApplicationCommand userCommand)
        {
            if (!IsAuthorized())
            {
                return new HttpResponseMessage(HttpStatusCode.Unauthorized);
            }

            var hasChanges = false;
            var user = await _db.Users.FindAsync(userCommand.UserId);
            if (user == null)
            {
                user = new User
                {
                    Id = userCommand.UserId,
                    FirstName = userCommand.FirstName,
                    LastName = userCommand.LastName,
                };
                _db.Users.Add(user);
                hasChanges = true;
            }

            var application = await _db.Applications
                .Include(x => x.Users)
                .SingleOrDefaultAsync(x => x.Name == userCommand.ApplicationName);
            if (application == null)
            {
                return new HttpResponseMessage(HttpStatusCode.NotFound);
            }

            if (!application.Users.Contains(user))
            {
                application.Users.Add(user);
                hasChanges = true;
            }

            if (!string.IsNullOrEmpty(userCommand.DefaultPermission))
            {
                var permission = await _db.UserAppPermissions
                    .Include(x => x.User)
                    .Include(x => x.Application)
                    .SingleOrDefaultAsync(x => x.Application.Id == application.Id &&
                                               x.User.Id == user.Id &&
                                               x.Permission == userCommand.DefaultPermission);
                if (permission == null)
                {
                    permission = new UserAppPermission
                    {
                        Application = application,
                        User = user,
                        Permission = userCommand.DefaultPermission
                    };
                    _db.UserAppPermissions.Add(permission);
                    hasChanges = true;
                }
            }

            if (hasChanges)
            {
                await _db.SaveChangesAsync();
                return new HttpResponseMessage(HttpStatusCode.Created);
            }

            return new HttpResponseMessage(HttpStatusCode.NoContent);
        }

        private bool IsAuthorized()
        {
            var key = ConfigurationManager.AppSettings["app:AuthorizationHeaderKey"];
            var expectedValue = ConfigurationManager.AppSettings["app:AuthorizationHeaderValue"];
            if (!string.IsNullOrEmpty(key) && !string.IsNullOrEmpty(expectedValue))
            {
                var headers = Request.Headers;
                if (headers.Contains(key))
                {
                    var headerValue = headers.GetValues(key).FirstOrDefault();
                    if (headerValue == expectedValue)
                    {
                        return true;
                    }
                }

                return false;
            }

            return true;
        }
    }

Sample web application

With all that in place we can finally switch to our client web application itself. The idea that this is one among many business properties, that requires consumer authentication and permissions from the API. The main reference used for putting this together was an Azure "quick-start", available as an open-source GitHub respository.

Key-points to mention include setting up certain areas of the web application to be protected, and requiring the user to register or sign-in to access them. With ASP.Net MVC, no matter what authentication method is used, this is done quite simply by decorating the appropriate controller action methods with an [Authorize] attribute. The user is then asked to authenticate using Azure AD B2C, based on the configuration supplied in StartupAuth.cs, and shown in this code file from the sample linked above.

The actual sign-in and registration pages are triggered via action methods as show below:

        public void SignIn()
        {
            if (!Request.IsAuthenticated)
            {
                // To execute a policy, you simply need to trigger an OWIN challenge.
                // You can indicate which policy to use by specifying the policy id as the AuthenticationType
                HttpContext.GetOwinContext().Authentication.Challenge(
                    new AuthenticationProperties () { RedirectUri = "/Account/Callback" }, Startup.SignInPolicyId);
            }
        }

        public void SignUp()
        {
            if (!Request.IsAuthenticated)
            {
                HttpContext.GetOwinContext().Authentication.Challenge(
                    new AuthenticationProperties() { RedirectUri = "/Account/Callback" }, Startup.SignUpPolicyId);
            }
        }

These methods differ from those in the sample only in that the redirect is sent to another action method on the AccountController, called Callback. This is responsible for then making the call to the permissions API to ensure the new user is recorded within the permissions database, with the default permissions for the application:

        public async Task<ActionResult> Callback()
        {
            // Get claims from the newly authentication user, and save to database via API
            const string UrlFormat = "{0}api/applications/{1}/users";
            var url = string.Format(UrlFormat,
                ConfigurationManager.AppSettings["app:IamApiEndpoint"],
                ConfigurationManager.AppSettings["app:AppName"]);

            var claims = ClaimsPrincipal.Current.Claims.ToList();
            var userId = Guid.Parse(claims.Single(x => x.Type == "http://schemas.microsoft.com/identity/claims/objectidentifier").Value);
            var firstName = claims.Single(x => x.Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname").Value;
            var lastName = claims.Single(x => x.Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname").Value;

            var command = new AssociateUserWithApplicationCommand
            {
                ApplicationName = ConfigurationManager.AppSettings["app:AppName"],
                DefaultPermission = ConfigurationManager.AppSettings["app:DefaultPermissionForNewUsers"],
                UserId = userId,
                FirstName = firstName,
                LastName = lastName
            };
            using (var client = CreateHttpClient())
            using (var response = await client.PostAsJsonAsync(url, command))
            {
                if (!response.IsSuccessStatusCode)
                {
                    throw new InvalidOperationException("Unexpected response from registration API request, status code: " + response.StatusCode + ", reason: " + response.ReasonPhrase);
                }
            }

            return RedirectToAction("Index", "Home");
        }

The other feature to demonstrate was to be able to retrieve the permissions configured by the administrator for the current user on the application. To illustrate this I've simply queried for these permission strings and rendered them to the screen on the user's profile page. Obviously in a real application they could be use for something more sophisticated, such as given access to further functionality or applying user-specific business rules.

        [Authorize]
        public async Task<ActionResult> ViewProfile()
        {
            ViewBag.Message = "Your profile and permissions.";

            var userId = ClaimsPrincipal.Current.Claims.Single(x => x.Type == "http://schemas.microsoft.com/identity/claims/objectidentifier").Value;

            const string UrlFormat = "{0}api/applications/{1}/users/{2}/permissions";
            var url = string.Format(UrlFormat,
                ConfigurationManager.AppSettings["app:IamApiEndpoint"],
                ConfigurationManager.AppSettings["app:AppName"],
                userId);

            using (var client = CreateHttpClient())
            using (var response = await client.GetAsync(url))
            {
                if (response.IsSuccessStatusCode)
                {
                    using (var content = response.Content)
                    {
                        var result = await content.ReadAsStringAsync();
                        ViewBag.Permissions = JsonConvert.DeserializeObject<IEnumerable<string>>(result);
                    }
                }
                else
                {
                    throw new ApplicationException("Unexpected response from API, status code: " + response.StatusCode + ", reason: " + response.ReasonPhrase);
                }
            }

            return View();
        }

Summary

As described earlier the work here was purely for a proof of concept and there's a number of areas where additional security and sophistication is needed for a production solution. Nonetheless the various parts, all hosted within Azure, all work together quite nicely to support a scenario where a number of different, user authenticated web properties can be used with Azure AD B2C, utilising the out of the box features and extending the permissions model with custom components.

Tuesday, 11 April 2017

Machine learning with F#

Another cross-post to something I've put together on my company's Medium site - in this article I'm discussing working with F# and decision trees (both new to me), in a what was an enjoyable, but futile, attempt to draw some insights from a data set of horse racing results.

You can read the article here.

Friday, 7 April 2017

Umbraco Show & Tell London

Recently had the pleasure of speaking with my colleague Ben Pilgrim at the Umbraco "Show & Tell" event in London. The theme of the day was the "great Umbraco editorial experience". There's a nice write-up of the day up on the Umbraco blog and we've made the slides available here.

Wednesday, 11 January 2017

Building an Alexa Skill with ASP.Net Web API

Another cross post of a piece written for my company blog, on building an Alexa skill. To read, click here.