That Conference 2016 – From Inception to Production: A Continuous Delivery Story

That Conference 2016, Kalahari Resort, Lake Delton, WI
From Inception to Production: A Continuous Delivery Story
Ian Randall

Day 3, 10 Aug 2016

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2016 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • Ian shares best practices of his development group at Pushpay, from inception to deployment

Tell a continuous delivery story in context

  • Pushpay company, SaaS business
  • ACMR Growth – committed monthly revenue stream
  • to $10 million in 3 quarters (5 considered fast)
  • Everyone at Pushpay paid to deliver value to business
    • Code not yet running on production server has no value

Hierarchy

  • Tools (top)
  • People, practices
  • Just culture & blameless postmortems
    • Bottom layer, most important

Our journey begins

  • Idea

Why?

  • Shared vision over the value to the business
  • Talk about why to build it
    • Better than talking about what

Who?

  • Who has conversation?
    • Product
    • QA – must involve QA in initial discussion
      • Can’t just test quality at end
      • “Bake quality in”
      • Build with tester (pairing)
    • Dev

Building a feature

  • Dev – how will I build this thing?
  • QA – how will I break this thing?
    • Very valuable insight
    • “Nobody will ever do that”
    • QA tester writes out what tests should be
    • Then Dev writes unit tests based on this
    • Quality Assistant (not Assurance)

Building a larger feature

  • Long-lived feature branch (git)
    • Delta gets too big
      • Smaller deltas are lower risk
    • No feedback – from user
    • (good) dry code
  • Feature switches (toggles)
    • Small deltas
    • Regular feedback
    • (bad) technical debt
      • Some code duplicatoin
      • But you have to go back later and yank out old code

Pushpay terminology

  • Delta – diff between production server & head of master branch
    • Code no yet running in production
  • Want to keep delta small and contained
  • Shipping in tinier increments, easier to figure out what the cause of the problem is

Feature Switches

  • Configuration per environment
    • Features.config
    • Features.Production.config
    • Features.Sandbox.config

Feature Switches

  • Url manipulation to toggle switches on/off
  • Deliver daily increments of (non-running) code
  • Light up a slice of feature
  • Measure – statsd
  • Re-think road-map to compmlete feature

Womm

  • Works on My Machine
  • Context switching after QA person finds problem
  • Pushpay, turned around
    • Must work on your machine
    • Tester decides this
    • Hand laptop to tester, let them test on the dev machine
    • Best value for company from dev point of view–watch tester break it right in front of them
    • Shortens the DEV/QA cycle
    • Pair testing

Code review

  • Every line of code gets reviewed
  • Code must reviewed and wommed before merging
  • “Roll forwards to victory”

Code Review

  • Do
    • Validate approach
    • Performance, security, operability
    • Cohesion, coupling
    • Be honest
  • Don’t
    • Be rude – e.g. “dude that’s gross”
      • Better – tell coder how you would have done it?
    • Seriously, don’t be rude
    • Sweat the small stuff, like bracing, spaces
      • This stuff is not important

Cross-Pollination

  • Someone else does it all again!
  • Pollination – not necessarily from your cell
  • Might not fully understand the context of your feature

4 Cotinuouses

(1) Continuous Integration

  • Source control
    • PR branch
    • Pushed early, for discussion
  • Build & Test
    • TeamCity does builds, using nUnit for unit testing
    • Integration testing – anything that crosses a boundary

CI – Source Control

  • PR-based workflow
  • Review happens in the PR
  • Everything reviewed

CI – build and test

  • PR Branch: Build, unit test and integration test
  • Merge into master – build, unit test, integration and acceptance test
    • Goes to QA – then rolled back or pushed to production
    • Human decision to push to production
    • Acceptance tests in Selenium – a bit brittle, but have some value
    • Create static Model and push to View, then test
      • If it breaks there, it’s not because of back-end
      • Verified that your site is intact, visually
      • Renders view against snapshot – if diff, either a bug or you accept as new snapshot

(2) Continuous Deployment

  • Automatically build, package and deploy to QA
    • Octopus deploy (great tool)
  • Manually promote package to production
    • One button click (because of Octopus)

(3) Continuous Delivery

  • Operability
  • Value

CD – Operability

  • Exception logging
  • App logging (Log4Net)
  • App metrics (statsd)
    • Measure absolutely everything
  • Incident

CD – Value

  • Add incremental bits of value to the product
    • Need to think – is there maybe value in shipping a portion of the feature?
  • Measuring the effectiveness

(4) Continuous Improvement

  • Actively seeking out opportunities to improve
    • Fix broken windows
    • Leave codebase in better state than when you found it
    • Improve the process

Bots

  • Shipbot, Beebot, Salesbot
    • Many, many, many more
  • @C3PR in action
    • Catalog of little commands
    • People joining PRs together

Just Culture

  • sidney dekker
  • Retributive culture
    • Clarity between acceptable and unacceptable
  • Restorative culture / model
    • Focuses on learning from what went wrong
    • Safe to fail

Fear of breaking things will paralyze an organization

Toyota’s Five Whys

  • Keep asking why until you get to root cause of problem
  • Doesn’t work for Pushpay
    • No single thing that is root cause
    • And often turns into “who”

Blameless Post-Mortems

  • Talk about how to stop the thing from happening again
  • When?
    • When there is an opportunity
    • Often, after break to production
    • Or even when something brings QA server down
  • How?
    • If we had a meeting, the loudest person in the room would do the most talking
    • So we do this asynchronously in a Wiki
    • Coordinated in slack channel #morgue
    • Co-ordinated with person closest to the incident
  • What?
    • Four sections in report
      • Scenario and impact
      • Timeline – write in real-time
      • Discussion
      • Mitigation – make sure this type of thing won’t happen again
        • Actionable ticket in Jira, highest priority possible
        • Slipping feature is better than having the incident happen again

Fault

  • Easy for manager to say to staff–when stuff happens, it’s not your fault
  • Much harder to go to CEO and say that stuff just happens
    • Board reads every post-mortem

That Conference 2016 – Clean Architecture: Patterns, Practices, and Principles

That Conference 2016, Kalahari Resort, Lake Delton, WI
Clean Architecture: Patterns, Practices, and Principles
Matthew Renze – @matthewrenze

Day 3, 10 Aug 2016

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2016 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • A whirlwind overview of domain-centric architecture and discussion of its merits
  • Overview of CQRS architectures (Command and Query Responsibility Segregation)
  • Brief mention of microservices architectures

Story – Tale of two architecures

  • John/Jane, same expertise
  • Hire them to design two buildings, same purpose
  • John
    • Superstar, creates the architecture, hands off to builder
  • Jane
    • Starts by talking to inhabitants and crews
    • Then involved during construction, helping implement
    • Tweaks plan as necessary
  • Captures essence of difference between clean architecture and traditional

About Me

  • Consultant
  • Pluralsight
  • Msft MVP

Focus

  • Enterprise architecture
  • Line-of-business applications
  • Modern equivalent of 3-layer
  • Will talk about 6 key ideas

What is software architecture?

  • High-level – higher level than the code
  • Structure
  • Layers – vertical partitions of system
  • Components – sub-divisions of layers, or horizontal
  • Relationships – how wired

Levels of architectural abstraction

  • System
  • Sub-systems
  • Layers
  • Components
  • Classes
  • Data and methods

Messy vs. Clean Architecture

  • Bad – like spaghetti
  • Good – lasagna
    • Nice layers

What is Bad Architecture?

  • Complex – accidental
  • Inconsistent
  • Incoherent – things don’t fit
  • Rigid – not adaptible
  • Brittle
  • Untestable
  • Unmaintainable

Clean Architecture

  • Simple
  • Understandable – easy to reason about
  • Flexible
  • Emergent
  • Testable
  • Maintainable – easier to maintain over lifetime

Clean architecture

  • Architecture that is designed for inhabitants of the architecture
    • Not for architect
    • Or for machine
  • Start designing architecture for inhabitants
    • This is the primary concern
    • Everything else is secondary

Why is clean architecture important?

  • Cost/benefit
  • Minimize cost to maintain
    • Good to optimize for maintainability
    • Since most cost is spent during maintenance
  • Maximize business value

Decisions

  • Context is king
    • Answer always depends on context
  • All decisions are a tradeoff
  • Use your best judgment

Domain-Centric Architecture

  • Heliocentric was more elegant model of the solar system

Classic 3-layer architecture

  • Database at central, then data access, business logic

Domain-centric

  • Domain at center
  • Application around that
  • Then Application
  • Database hung off the side

Uncle Bob

  • Most important thing is that the architecture is usable

Essential vs. Detail (e.g. for house)

  • Space is essential
  • Usability is essential
  • Building material is a detail
  • Ornamentation is a detail

Essential vs. Detail (Clean architecture)

  • Domain is essential
    • Domain: series of object models at center
    • Mirrors the mental models of the domain
  • Use cases are essential
    • And users
  • Presentation is a detail
  • Persistence is a detail
    • Can store in relational DB, NoSql, doc DB, etc.

Back to domain-centric diagram

  • What is essential is at the center of the diagram
  • Domain essential
    • At center
    • Everything points towards the domain

Hexagonal architecture

  • Application layer (domain) at center
  • Adapting to presentation
  • Adapting to do input
  • Adapting to persistence

Onion Architecture

  • Domain Model at center
  • Services
  • UI at outer

The Clean Architecture (Uncle Bob’s)

  • Entities at center
  • Use Cases
  • Controllers
  • Outside: Devices, Web, UI, External Interfaces

It’s all the same thing

  • All three of these are fundamentally the same
  • Domain at the center

Why use domain-centric architecture?

  • Pros
    • Focus on essential
    • Less coupling to details
    • Necessary for DDD
  • Cons
    • Change is difficult
    • Requires extra thought
    • Initial higher cost

Application Layer

  • Might embed use cases as high-level logic

What are layers?

  • Levels of abstraction
  • Single-Responsibility
  • Developer roles/skills
  • Multiple implementations

Classic 3-layer architecture

  • Database at bottom
  • Data Access
  • Business Logic
  • UI
  • Users
  • Top layers dependent on layer below it

Modern 4-layer architecture

  • Presentation at top
  • Application below that
  • Domain layer
    • Only domain logic
  • Persistence below application
    • Access to DB
  • Infrastructure below application
    • Access to OS

Application layer

  • Implements use cases
  • High-level application logic
  • Knows about domain layer but not persistence or infrastructure
  • No knowledge of upper layers

Layer Dependencies

  • Dependency inversion
  • Inversion of control
    • Details depend on abstractions
  • Independent deployability
  • Flexibility and maintainability
  • Flow of control
    • Presentation calls Application
    • Persistance calls Application

Example

  • Presentation layer w/controller
  • Application
    • Command dep on IDatabaseContext, IInventoryClient
    • Application–what the users are doing with the system
      • E.g. ICreateSalesCommand
  • Dep on Sale in Domain
    • Domain–objects in system
      • e.g. Sale object
  • Interface in cross-cutting concerns

Why use an application layer?

  • Pros
    • Focus is on use cases
    • Easy to understand
    • Follows DIP – Dependency Inversion Principle
  • Cons
    • Additional cost
    • Requires extra thought
    • What is application logic vs. domain logic ?
    • IoC is counter-intuitive

Commands and Queries

  • Keep separated
  • Command
    • Do something
    • Modifies state
    • Should not return value
  • Queries
    • Answer question
    • Do not modify state
    • Always returns value
  • Why?
    • Avoid side-effects
  • Sometimes odd, exceptions
    • E.g. create record, return it

CQRS Architectures

  • Application layer, separate into two parts
  • Left
    • Queries
    • Data Access
  • Right
    • Commands
    • Domain
    • Persistence
  • Database at bottom
  • Data flow
    • Down through commands
    • Up through queries
  • CQRS is domain architecture done in sensible way

CQRS Type 1 – Single Database

  • Single database, typically NoSQL
  • Persistence layer might be something like EF
  • Queries – stored proc, linq to sql

CQRS Type 2 – Read/Write Databases

  • Command stack leads to Write Database
    • 3NF
  • Read database
    • 1NF, optimized for reads
  • Use “eventually consistent” model to push data across from write to read
  • Orders of magnitude performance improvement over single database
    • Because we mostly do reads, not writes

CQRS Type 3 – Event Sourcing

  • Command stack leads to Event Store
  • Read database under query stack
  • Replay events to get current state of entity (from deltas)
  • State modifications pushed over to Read Database

Type 3 points

  • Complete audit trail
  • Point-in-time reconstruction
  • Replay events
  • Rebuild production database
    • Just by replaying events
  • Only worth doing if you need these features

Why use CQRS?

  • Pros
    • More efficient design
    • Simpler within each stack
      • At expense of inconsistency across stacks
    • Optimized performance
      • Of each side
  • Cons
    • Inconsistent across stacks
      • More complexity
    • Type 2 is more complex
      • Adds consistency model
    • Type 3 might be overkill
      • If you don’t get business value from these features

Functional organization

  • Screaming architecture
    • Architecture should scream the intent of the system
  • Organize architecture around use cases
  • Uncle Bob

Building metaphor

  • House, typical
    • Intent – residential
    • Rooms embody uses–bedroom, kitchen, living room
    • Architecture shows use
  • Look at list of components, rather than architectural diagram
    • Can’t infer intent

Software intent

  • Could organize folders by components (models, views, controllers)
  • Or organize by things
    • Customers, products, vendors

This vs. that

  • Traditional
    • Content, Controllers, Models, Scripts
  • Functional
    • Customer
    • Product

So what?

  • Functional cohesion is more efficient
    • Because it better models how we think about the software

Why use functional organization

  • Pros
    • Spatial locality
    • Easy to navigate
    • Avoid vendor lock-in
  • Cons
    • Lose framework conventions
    • Lose automatic scaffolding
    • Categorical is easier at first

Microservices

Components

  • Horizontal – UI, business, data access
  • Vertical – Sales, Support, Inventory
  • UI presents all vertical pieces as single unified UI

Problem Domain

  • Sales and Support

Single domain model

  • Traditionally, create single domain model
  • E.g. Product on both sides, so we have single Product entity
  • Employee – sales or service
  • Problem
    • Becomes exponentially more difficult

Bounded contexts

  • Sales and support as boundaries
  • Some entities in just Sales
    • Some in Support
    • Some in overlap
  • Then pull apart and have separate models for each context

Microservice architectures

  • Subdivide system
    • Communicate with each other via lightweight mechanisms
  • Bounded contexts
    • Can have one agile team per bounded context
    • Only have to know about one bounded context
  • Small teams
  • Qualities
    • Independent
    • Similar to SOA
    • Independently deploy and scale each microservice
  • Size of microservices
    • Ongoing debate
    • Bounding context maps to microservice
    • So persistence model matches context matches service
    • “Goldilocks” point

Why use microservices?

  • Pros
    • Less cost for large domains
    • Smaller teams
    • Independence
      • Don’t need to know what other teams are doing
  • Cons
    • Only for large domains
      • Overkill for small domains
      • Lot of overhead cost for microservice
    • Higher up-front cost
      • Fault tolerance, latency, load balancing
      • Might start with single system, split when you have multiple bounded contexts
      • – Distributed system costs

Code Demo

  • Solution architecture for simple web site
  • ASP.NET MVC 5
  • Top-level folders
    • Application, Common, Domain, Infrastruccture, Peristence, Presentation
  • Presentation, Sales folder
    • sub-folders Models, Services, views
  • Application layer
    • Sales | Commands, Queries
    • Under Commands, folder for each command
  • Domain layer
    • Fairly thin object model
    • But a small amount of business logic in the objects

Where to go next

  • Patterns of Enterprise Application Architecture – Fowler
  • Bliki has updated stuff
  • Clean Code – Uncle Bob
    • cleancoders.com
    • Great
  • Domain-Driven Design – Evans
    • For complex domains
  • Greg Young, Udi Dahan
    • CQRS event sourcing
  • matthewrenze.com
    • Pluralsight courses as well

That Conference 2016 – Lean UX

That Conference 2016, Kalahari Resort, Lake Delton, WI
Not Just Arts & Crafts: A Developer’s Guide to Incorporating Lean UX Practices into the Devleopment Process
Rachel Krause

Day 3, 10 Aug 2016

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2016 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • An argument for a lean/agile approach to UX design

Contact Info

@rachelbabiak // #thatconference
centare
– On site, on SCRUM teams
– Consulting, be UX person on client team
– UX coaching

UX / UI

  • Not just putting a pretty picture on something
  • UX is the experience that the user goes through
  • What will help the user in the best way possible

Example: Dominoe’s Pizza Tracker

  • Visually, see where pizza order is in the process
  • Distracts the user a bit while they’re waiting
  • Really helped Dominoe’s competitively

UX Schools of Thought

  • Genius Designer
  • Mindset of experimentation

Genius Designer

  • Research, work done up front
  • Outside of development team
  • Developers have no say
  • “Best design possible”
  • If something goes wrong, blame the user

Mindset of experimentation

  • No matter how brilliant you are, you can’t figure it out
  • Release early/often

Lean UX

  • Book: Jeff Gothelf
  • Focus on experience being designed
  • Move us forward

Focusing on 3 principles from book today

  • Accept failure
    • It’s okay to be wrong
  • Reduce waste
    • Get rid of high fidelity mockups
  • Collaborate
    • Everyone gets a say in the design
    • Gets everyone on the same page

Trying to fit Lean UX into Agile

Explain via cake metaphor

  • Big Design
    • Pretty picture of cake–you’ll get this at end of project
    • During development, you get cake that looks similar
    • Pivot when necessary, make changes
  • Lean UX
    • Start with wish list, rather than picture
    • Everything about the list can and will change
    • Scary to think of UX as just a list of features
    • Start with something small, e.g. cupcake
      • Everything there, but on smaller scale
      • Will release to users
    • Users give feedback
    • Then come out with something bigger
      • More features
    • Eventually get to something bigger, but matching what users need
    • Or cupcake might be all that the users need

Example: Pokemon Go

  • Added feature after field testing
    • Take screenshot

Best UIs are designed by the entire team

  • Different roles bring different perspectives

Example: A website

  • QA might add opinion about usability
  • Dev weighs in
  • Product Manager weighs in
  • Design changes as people offer thoughts

No complicated software necessary

  • Nothing that you have to pay money for
  • Need to use something that will work for entire team
    • E.g. Just a whiteboard
  • No time for high value mockups
  • Don’t get too attached to mockups

Personas

  • Traditionally–heavily researched, validating
    • Packaged up and given to stakeholders
    • This takes a lot of time

Proto-Personas

  • Easier, lean
  • Originate from brainstorming sessions
  • Based on stakeholders’ domain expertise
  • Starting point for understanding users
  • Think about role
    • E.g. Administrator, Warehouse Worker, Read-only User

Demo: Proto-Personas

  • Four quadrants
  • Upper left–name and face
    • Gives up empathy for the user
  • Upper right–basic biographical information
    • Who, where coming from, what devices
    • Can be specific, e.g. married with 2 kids
  • Lower left–pain points & needs
    • Frustrations with current solutions / products
  • Lower right–potential solutions
    • Can be subjective, e.g. wants things super easy
  • Do this on whiteboard, can take photo
  • Stakeholders invited in on this process
  • Talk through things and zero in on who you’re building software for
  • If you can validate with real people, your assumptions were correct

Kids Experiment

  • Had kids run through this process, with superheroes

User Journeys

  • Once you have persona, give him a journey
  • Series of steps that user is going to go through when working with software
  • [Persona] wants to [action] because [need] but [friction]
    • James wants to check in an inbound shipment because a railcar just arrived, but he doesn’t have accurate weights yet.

Now map the process

  • Steps in user journey
  • What happens, what the user does
    • Can do the journey for pre-solution or post-solution
  • Needs, activities and expectations
    • At each step
  • Persona’s emotional state
    • Graph between + (delightful) and – ()
    • Give rating for each step in the journey
  • Opportunities for Improvement
    • At each step
    • Don’t worry about scope at this point
  • Can do in Google Slides
    • Something simple

Design Sessions

  • In place of Photoshop stuff
  • Get an idea of the layout
  • On whiteboard
    • User needs, check things off as you address them
    • Have small journey map to see where you’re at
  • Do this collaboratively
  • Why do this
    • We might think we’re all on the same page
    • Once we draw it out, we realize we’re thinking differently
    • Then hash through things and come to the same vision

When?

  • Don’t have to do all of this than every single thing
  • Proto-personas
    • At start of project
    • Or later, if you don’t already have them
  • User Journeys
    • Whenever you don’t have enough information to fill out acceptance criteria
  • Design Sessions
    • When you need to be on the same page
  • Don’t do extra meetings for this
    • Can piggyback on existing Scrum meetings

Example: salesforce ux

  • Wanted to redesign their core web product
  • Had 85 designers and 60 Scrum teams
  • Used Lean UX to revisit assumptions
  • Then collaborated, included everyone on the team
  • Lightweight, just using Sharpie
  • How, with so many people?
    • Regular UX reviews on the calendar
    • Developers in design process early and often

Got a story?

  • Let Rachel know successes and failures

Info

  • @rachelbabiak // #ThatConference
  • linkedin.com/in/rachelkrau
  • speakerdeck.com/rachelkrau

Questions

  • Remote teams?
    • Some tools out there for virtual design session
    • E.g. Peer.in, Google Slides
  • Comment – When documenting, ask why you’re creating the document
  • Comment – Balsamic
    • Cartoony layout keeps people from focusing on the details
    • Rachel: Challenge that, since it’s a paid application
      • Why not just use whiteboard
  • What to during design process
    • Translate some of the needs and pain points into design elements
    • Not a formal process, but doing gut check w/user’s pain points
  • Examples of turning negatives in journeys into positives
    • Often quick wins, e.g. confirmation dialog to avoid unintentional deletes
    • For bigger ones, you talk about how much value is in an improvement
  • How do you validate a design after it’s out there?
    • Identify who your users are, then do beta test
    • Do beta testings with stakeholders and users
    • Watch them walk through the journey, validate
    • If you have no users, you’ll have to make assumptions
    • Might go down hallway and use someone else, e.g. product manager

That Conference 2016 – Daring to Develop with Docker

That Conference 2016, Kalahari Resort, Lake Delton, WI
Daring to Develop with Docker – Philip Nelson, Omni Resources

Day 2, 9 Aug 2016

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2016 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • Progression from bare metal to containers
  • Demoing Docker

Contact Info

github.com/panmanphil
twitter.com/panmanphil
panmanphil.wordpress.com

Docker and Containerized Apps

History: Bare Metal, VM, Docker

Racks of boxes

  • Manually installed hardware, software
  • Must configure everything
  • Deplyment lightly automated

Virtual machines

  • Software installed servers (guests)
  • Still manually configure
  • Lightly automated deployments
  • This is an improvement
    • Easier to provision hardware, servers, networks
    • Mainly helps IT admins

Devops

  • Intersection of Development, QA, Operations

Devops 1.0

  • Improvement: automated setup for your dev environments
  • E.g. Vagrant, Chef, Puppet, Ansible
    • These tools set up OS
  • Provisioning: setting up everything that you need for running your application
  • Drawbacks, trouble spots
    • Don’t always handle dependencies
    • Not quite complete
    • Took maybe 15-20 mins to set up box
    • Eventually, the host OS gets messy
  • Provisioners only useful at start for clean boxes
    • They don’t delete stuff

Containers

  • Slice VM up into smaller units
  • Automated OS setup, automated application setup, automated deployment
  • Each unit from application perspective, appears to own the OS
    • In reality, units share same kernel

Containers vs. VMs

  • VM
    • Each VM has guest OS, with shared libs and app
  • Container
    • Docker Engine on Host OS
    • Libraries on top, shared libraries across all applications
    • Then multiple instances of App on top of each set of shared libs

Docker Interactions

  • Containers
    • Commit (to image)
    • Run (from image)
  • Images
    • All files, code + binaries
    • Pull to Images from Registry
    • Only pull what’s different
    • Push to Registery after building

Containers – Building

  • Build machine
    • Pulls from git
    • Build app.jar
    • Upload to Docker Hub
  • Docker Hub
    • Run on production machine–docker run <app image>

Containers – Hosting Prior to 8/2016

  • Locally, create VM for running containers in
  • docker-machine create –driver virtualbox thatconference

Containers – Hosting Now

  • Containers run in native hypervisor on OS X or Win 10
  • Can still use docker-machine and separate collections of images, but most won’t do this

Containers – Running

  • You run with an image
  • docker run -d yourregistryname/yourimage
  • Map ports to host ports
  • Use volumes to persist data between different versions of image
    • Docker containers are immutable
    • Can save data that your application is running by using volumes

Containers – Demo

  • docker run -d mongo:latest
  • docker ps
    • What’s running on this box
    • Assigns random name to box
    • More detail with container id
  • docker logs namehere
  • docker inspect namehere
    • .json document that talks about how it was set up
  • Container is just a definition of what should run
  • Stopping
    • docker stop namehere
    • Containers not yet dead
  • docker ps -a
    • Containers stick around
  • Remove permanently
    • docker rm namehere

Run on Windows

  • Docker runs on both Win 10 and OS X
  • bash shell stuff a bit problematic on Windows
  • docker run -it hello
    • .NET Core on Debian
  • File changes not quite working–compiled even after no changes

Link Node.js container to Mongodb from command line

  • Networking established by container, no need to map external ports
    • Injects name of other containers
  • Run Mongo, run node-starter
    • -e MONGODB=mongodb://mongo:27017
    • –link mongo
  • Link will link to the other container

Containers – Volumes

  • Data will stay with running container only when it is defined, running or stopped
    • So if you start/stop the same container, the data will be there
  • But container definition is immutable
    • So data will go away if you remove container and start again
  • Volumes tell docker to manage longer term persistence of data independent of the container
    • As long as one guy is pointing to volume, it will remain
  • Use -v on command line

Demo – Volumes

  • docker run -it -docker-data –volumes-from mongo

Containers – Volumes

  • For working locally on rapidly changing code, can map any folder in container to host volume
  • Use -v command to map volume

Containers – Registry

  • Build image using docker build
    • docker build
  • After image is built, can be launched locally

Containers – Swarm

  • Make cluster out of docker machines and/or host machines
  • Docker swarm is part of base docker release
  • Some competitors, e.g. Amazon ECS
    • ECS “not that wonderful”

Containers – Troubleshooting

  • inspect
  • logs
  • docker ps
  • docker images
  • Log into running container
    • docker exec -t <name> /bin/bash

That Conference 2016 – Introduction to Angular 2.0

That Conference 2016, Kalahari Resort, Lake Delton, WI
Introduction to Angular 2.0 – Jeremy Foster, Developer Evangelist at Microsoft

Day 2, 9 Aug 2016

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2016 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • Choosing Angular 2
  • A few core concepts of Angular
  • How to create a site and add a component
  • Streams

Contact Info

@codefoster
codefoster.com/thatdeck
github.com/codefoster/waterbug-ui

Choosing Angular 2

  • It really matters what you choose, what you depend on
  • One way of comparing frameworks is to look at trends, google trends, i.e. see what people are searching on
    • Ember – peaked and then decline
    • Backbone – bit older, didn’t peak as high as Ember, also dropping off
    • React – Much higher than previous 2, heading up sharply
    • Angular – dwarfs even React (1 and 2)
      • No decline
      • In fact, there’s an uptick at end (recently)
  • Good to find a library that people are using

Virtual DOM

  • Angular does this
  • Doesn’t directly manipulate DOM
  • Creates virtual from real DOM
  • Manipulates virtual DOM
  • Diffs/updates the real one when it matters
  • Dramatic performance increase
  • Less time spent in CSS rendering

Component Model

  • (also done in Angular 2)
  • Evolution: Pages to MVC to Components
  • Easier to reason about app structure (node on tree)
  • Excellent encapsulation
  • Excellent reusability
  • Forward compatible with components standard

Web architecture paradigms

  • Pages, MVC, Components
  • Enter app
  • Pages: at a page
  • MVC: Enter at controller
  • Components
    • Tree structure

Favorite things about Angular.js

  • Virtual DOM, component architecture
  • Cohesive, relatively complete framework
  • TypeScript
    • No more friction to turn into JavaScript
    • Can easily do new ES6 things
  • Rx.js for streams
  • Great support and community

Typescript

  • Beautiful
  • Valid Javascript is automatically valid Typescript
  • Can just rename .js to .ts
  • Can learn one concept at a time
  • Write Typescript, transpile into Javascript
  • Types
    • Compiler gives a bunch of help
    • Lots of pain goes away
  • String literal
    • Easily embed variable values
  • typescriptlang.org/play
  • TypeScript gives us types, interfaces
    • Just create interface, duck-typing, don’t need class

Easiest Way to get Started in Angular

  • npm install -g angular-cli
  • ng new myapp
  • Makes folder
  • Takes a while, doing npm install for new app
  • Nearly ready for production at this point
  • But takes some reverse engineering to figure out how everything is wired up
  • Has tests
  • Using Broccoli for task running
  • src folder has your code, built into dist folder
  • ng serve
    • Fires up port, transpiles TypeScript
    • In watch mode, in case something changes
  • ng generate component hello-world
    • Create new component
  • ng generate service greeter

Resources

  • angular.io
    • Quickstart
    • Tutorial
    • Developer’s Guide
    • Cookbook
  • cli.angular.io
  • github.com/angular/material2 (in beta)
    • For skinning, use material design elements
    • Looks like Google’s material design spec
  • codefoster.com/github/waterbug

Components

  • Tree of components
  • These are components of UI
    • Although some may not show up visually (e.g. script)
  • E.g. App, NavBar, Header, Body, etc.
  • Circular references are ok
  • Components can be ref’d by more than one parent
  • Flip switch and you’re creating new web component

Demo: Adding a component

  • From command line
    • ng generate hello-world
  • You’re sitting inside existing angular app
  • Gives you test for component
    • Tests travel with their components
  • .ts files (like .js)
  • Look at this in IDE
    • Use Visual Studio Code
    • WebStorm is alternative, but Code better for TypeScript
    • Even Google team is using Visual Studio Code
  • We see new hello-world folder
  • Look at .ts code
    • import–like require
    • Can bring in multiple pieces from single library, e.g. angular\core
  • @component is basically a class
  • @component decorator block
    • moduleId: allows using relative paths
    • Compilation error because we need to switch to common.js
    • Turns class into component based on decoration
    • selector – dictates tag, e.g. “app-hello-world” => <app-hello-world>
      • Set to hello-world
      • Can do conditional tags
    • templateUrl: hello-world.component.html
      • HTML that will contain this component
      • Could also inline the HTML, but generally worth having separate HTML
    • styleUrls: hello-world.component.css
      • Could have more than one
  • class
    • Constructor and onInit
  • Add hello world stuff
    • In .html

Run app

  • ng serve
  • not yet instantiating the component
  • Add component to app
    • app.ts
    • import { HelloWorldComponent } from ‘./hello-world/hello-world.component’;
    • On @Component, do new directive
      • directives: { HelloWorldComponent }

Services

  • Dashed circle in component tree
  • Just a class that you’ll use
  • Cascades
    • Everything below a service in the tree can use it
    • Service will be Singleton for everything under it
    • Could bring in at top, but not a great practice

Create service

  • ng generate service greeter
  • Create simple service that lets you ask for a message
  • Look at generated code
    • class GreeterService
    • Create new function
    • getMessage() { return “howdy”}
  • Call the service
    • import { GreeterService } from “../greeter.service”
    • Add provider in @Component: providers: {GreeterService}
    • Dependency inject into your constructor
      • Private modifier on parameter makes it available throughout rest of class
    • this.greetsvc.getMessage();
  • Make data available to your view
    • Could create local property and set property to result of call
    • Anything in class can be used in view
  • Update view
    • hello there {{ message }}
    • (where message is member property)
    • Lots of other richness for inserting data in the view

Debugging

  • Can do a lot of debugging in the browser
  • Augury as Chrome extension
  • Can give you Angular info at runtime

Streams

  • Let’s visualize imperative programming
  • Left to right, do one thing after another
  • Function call, comes back with something
  • Grammar review
    • Declarative: I’d like it if..
    • Interrogative: would you please do this
    • Imperative: fetch me a beer
  • Problems with this model
  • Better to build rule than build task
  • Want to do asynch, do something while waiting for function call to return

Imagine four quadrants

  • X Axis–Scalar vs. vector data
  • Y Axis–Sync vs. async
  • Sync
    • Vector: array
    • Scalar: T
  • Async
    • Scalar: promise
    • Vector: streams go here

Dealing with collections of things

  • Return multiple things back from asynchronous function
  • Sometimes array is huge and we get out of memory issue if we return everything all at once
  • Could use Generator
    • Hard to understand
    • Concept is function that can return in middle, giving you a few things at a time

Back to asynchronous problem

  • Callbacks not that great
  • You have to specify things up front

Promises are nicer

  • Gives us thing back right away
  • Chain well and form an asynchronous pattern

But generators and promises are not quite enough

Promises are a pain

  • Only one return value
  • Can’t cancel them
    • E.g. open web call that you can’t cancel

Array of promises?

  • Takes a lot of babysitting
  • Hard to orchestrate
  • Difficult to handle timing

[Sean: Session ran long and I had to leave at this point]

That Conference 2016 – C#: You Don’t Know Jack

That Conference 2016, Kalahari Resort, Lake Delton, WI
C#: You Don’t Know Jack – George Heeres

Day 2, 9 Aug 2016

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2016 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • This talk presented a series of lesser known but useful C# skills

Strings

  • ANSI C
    • Array of characters
    • Ptr to null-terminiated array of chars
  • Getting length
    • More efficient to put length at start of string
  • Appending
    • Can’t just append in-place
    • Allocate new array, copy both pieces

C# Strings – What not to do

  • Long series of + operators to build up a string
  • Luckily, compiler makes this more efficient
  • A CIL diversion
    • Common Intermediate Language (aka MSIL)
    • Runs on CLI
  • 2nd example for concatenation
    • while loop that does + operator to append each piece
  • Better–use StringBuilder
    • Give it suggestion for size, should make best guess
    • Default capacity on StringBuilder is 16
    • Then doubles whenever it runs out of space

IFormatProvider

  • Format specifiers, built-in and custom
    • Putting object in format string, invokes ToString()
  • By default, ToString() returns type name
  • Can create custom IFormatProvider
    • E.g. if you want some custom format for telephone numbers
    • Bad–can’t globally register

String interpolation (C# 6)

  • Simplifies format string
  • Avoid problem with bad index
  • $”some test (varname) more string”
  • Compiler converts this to classic string format

Debugging helper

  • Attribute – DebuggerDisplay, give format string to indicate how to format in debugger

Operators: Ternary

  • ? : syntax
  • condition ? true : false
    • Return null if lefthand portion evaluates to null
    • Never nest ternaries
  • C#6: dice?.Values

Null coalescing operator

  • left ?? right
  • Return left if operand non-null, else return right
  • Good for lazy loading properties
    • return _dice ?? (_dice = init())

Operators: Implicit/Explicit

  • Write implicit operator, e.g. convert object to int, perhaps person.id
  • Allows assignment without cast operator

Operator overloading

  • Standard arithmetic operator

Enumerable: yield return

  • In method for IEnumerable, only execute code when the enumeration is traversed
  • Technically, violates MVC pattern
  • Worry about whether connection is still open at a later time
  • Bypass this–calling ToArray, ToList forces fetching all objects

Bit math |, &

  • Use in enumeration, set flags as power of twos
  • Can then store multiple values in single enumeration variable
  • Need [Flags]
  • Can create helper values in enum, combination of other enumerated values

Constructors – DRY > “Daisy Chain”

  • Don’t do the same initialization across multiple constructors
  • DRY – Don’t Repeat Yourself
  • Invoke another constructor with this keyword
  • Can go up or down (fewer or more parameters)

Extension methods

  • Just syntactic sugar
  • But very useful

Hacking: Decompiling Code

  • JetBrains et all
  • Decompile from IL, i.e. creates C# from IL

Hacking: Oh Snap

  • By default, anybody can decompile from your IL
  • Mitigate
    • Tools
    • Don’t store sensitive data
    • Move algorithms to server
    • .NET Obfuscator

Reflection

  • Review: can’t access private data
  • Can actually access using reflection
  • Example
    • GetType(), GetProperty(), GetValue()
    • BindingFlags Instance, NonPublic
    • Can also call method
    • Can also reflect on static data
      • Just need BindingFlags.Static
  • Generics
    • Get initial type, then construct generic type
    • GetConstructor or do MakeGenericType

Reflection: A Practical Example

  • Adding DisplayName attributes on enum values
  • This is a custom attribute
  • Then write extension method for the enum type, to get the display name
  • GetCustomAttributes on the type, find the attribute, cast

Crafstmanship: Refactoring

  • Best practices
  • Always leave code better than when you found it

“Magic” numbers

  • Move magic numbers to const int, define in just one place

Be expressive & declare intent

  • Move check into method, with name that tells you intent
  • E.g. IsValidTelephoneNumber()
  • Encapsulates ugly code, cleans up main code that you’re reading
  • Good code is self-documenting

Refactoring: String comparisons

  • Case insensitive comparison
  • ToLower or ToUpper–inefficient
  • Better
    • Use string.equals, specify StringComparison.OrdinalIgnoreCase

Refactoring: Intellisense

  • To add intellisense for your code, use XMLDoc in front of everything
  • [Sean: Also shows up in object explorer]

List<T> Properties

  • Returning List<T> or IList from class
    • Implies functionality present in IList that you may not have implemented
    • Instead, return IEnumerable if you’re returning read-only list

Extensions: DTO

  • In n-tier model, use DTO (Data Transfer Object)
  • Example, same DTO going from database all the way out to HTML
  • Problems
    • Leaky abstraction
    • If you change data access layer, changes go through every layer
  • Better
    • New data object at each layer
    • Tools like structure map can help map from one object to another
    • Or just use extension methods to convert from A to B
    • Remember–layer should not know anything about layer above it
    • Extension methods help with this
    • Conversion is in the outer layer, not the inner

Extensions: More.. Support IEnumerable

  • Take IEnumerable a, return IEnumerable b
  • Use yield
  • Or write with LINQ

That Conference 2016 – What’s that Smell? It’s your Entity Framework!

That Conference 2016, Kalahari Resort, Lake Delton, WI
What’s that Smell? It’s your Entity Framework! – Russ Thomas

Day 2, 9 Aug 2016

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2016 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • Entity Framework often gets a bad rap when it appears to generate very poor T-SQL
  • You can generate far better T-SQL by paying attention to how you construct queries in LINQ
  • This talk describes several “code smells” that lead to poorly performing T-SQL and that can easily be improved
  • [Sean: Most of this also applies to LINQ to SQL]

All communication with SQL Server is via T-SQL

  • T-SQL is the only way to talk to the server
  • From T-SQL to database storage
  • Goes through Optimization (indexes, etc.)
  • There’s no such thing as a technology that bypasses T-SQL
  • Entity Framework adds 4th piece
    • EF -> T-SQL -> Optimization -> Database Storage
  • EF makes things easier because it quickly scaffolds T-SQL
  • Goal today–work towards higher performance in EF

Code smell #1 – too many columns

  • EF can bring back too many columns from database
  • Demo
    • Retrieve data from People table, do something with LastName, Firstname
    • Using simple LINQ statement, just do select p (e.g. from person table)
    • Then foreach on results, pull off LastName, FirstName
  • Look at what EF sends
    • Start up Profiler (yes, should use Extended Events)
    • Reminder: never do the trace in production
    • If writing this query manually in T-SQL, we’d do simple query
    • Look at trace from EF version of this
    • Query is selecting large list of columns–far more than we’re using
    • Can also look at Reads metric in Profiler
  • Recommendation
    • Specify exactly the columns that you want
    • Use Projection
    • instead of: select p,
    • use: select new {p.LastName, p.FirstName}
    • Goes from 544 reads to 4 reads
    • Query now looks like what we’d write ourselves

Code smell #2 – Non Set Based Work

  • Demo–Retrieving e-mails
  • Write a couple of loops, loop through People p, then loop through p.EmailAddresses (implicit join)
  • LINQ query is still returning entire People table
  • And then code is iterating through everything to get e-mail addresses
  • This results in a new T-SQL statement for each pass through the loop
  • Horrible performance
  • Don’t retrieve data procedurally
  • Relational DB works better if you retrieve using set-based query
  • Solution
    • In T-SQL, you’d just do a join
    • In LINQ
      • from people in db.People.Include(“EMailAddresses”)
      • Tells EF that we’ll be coming back later for e-mail address
    • Even better way than .Include
      • Do join in LINQ
      • Then only a small iteration on final results

Code first discussion and demo

  • Most people prefer model-first, but code-first is slick
  • Can write code and then generate DB from code
  • Discussion
    • By default, it set column type to nvarchar(max) for string
    • And decimal(18,2) for double (e.g. a salary field)
  • Conclusion
    • IF you’re going to do code-first, go further than just column names and types
    • SQL Server does care about max string length, makes educated guesses on storage
  • Better way
    • Override OnModelCreating in DbContext, use fluent syntax on modelBuilder to set stuff up
    • Set various attributes for each column, e.g. maximum
    • Re-gen model, can see better attributes on columns
  • Thoughts
    • Code-first, often starts as simple hobby project
    • But you really do need to set proper attributes ahead of time

How to see what EF is doing under-the-covers

  • Profiler
    • Don’t run on production server
  • So how do we see what’s going on in EF without using profiler?
    • SqlLogger class, IDBCommandInterceptor
    • Requires EF 6
    • Can write out to text file, capturing TSQL statements
    • You implement different methods to capture different kinds of things
    • Can do xxxExecuting or xxxExecuted
    • Then: DbInterception.Add(new MyCommandInterceptor());
  • Stuff is intercepted client-side
    • So doesn’t impact performance on server
  • [Sean: Can also set context.Database.Log property]
  • Other stuff you can do with interceptors
    • Intercept and change statement, e.g. “soft deletes” where they don’t actually delete but just set IsDeleted column
    • No trigger !

Other comments

  • There are other optimizations you can do that have nothing to do with SQL Server

Wrapup

  • Will put zipped up demo on sqljudo.com