async

That Conference 2017 – Concurrent Programming in .NET

Posted by

0

That Conference 2017, Kalahari Resort, Lake Delton, WI
Concurrent Programming in .NET – Jason Bock (@JasonBock)

Day 2, 8 Aug 2017

Disclaimer: This post contains my own thoughts and notes based on attending That Conference 2017 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of the speakers.

Executive Summary

  • Doing concurrency correctly is very hard when doing it at a lower level
  • Don’t use Thread, ThreadPool anymore
  • Learn and use async/await


Background

  • Concurrency is tough, not as easy as doing things sequentially
  • Games are turn-based
    • But interesting exercise to do chess concurrently
    • Much more complicated, just as a game
  • We’ll just look at typical things that people struggle with


Terms

  • Concurrency – Work briefly and separately on each task, one at a time
  • Parallelism – People all working at the same time, doing separate tasks
  • Asynchronous – One you start a task, you can go off and do something else in the meantime


Recommendations

  • Stop using Thread, ThreadPool directly
  • Understand async/await/Task
  • Use locks wisely
  • Use concurrent and immutable data structures
  • Let someone else worry about concurrency (actors)


Stop Using Thread, ThreadPool

  • This was the way to do concurrency back in .NET 1.0
  • Diagram – Concurrent entities on Windows platform
    • Process – at least one thread
    • Thread – separate thread of execution
    • Job – group of processes
    • Fiber – user-level concurrent tasks
  • But mostly we focus on process and thread
  • Example–multiple threads
    • Use Join to observe at end
  • Problems
    • Stack size
    • # physical cores
  • # Cores
    • May not really be doing things in parallel
    • More threads than cores–context switch and run one at a time on a core
    • Context switches potentially slow things down
    • Don’t just blindly launch a bunch of threads
  • Memory
    • Each thread -> 1 MB memory
  • ThreadPool is an improvement
    • You don’t know when threads are complete
    • Have to use EventWaitHandle, WaitAll
    • Lots of manual works
    • At least threads get reused
  • Existing code that uses Threads works just fine


Async/Await/Tasks

  • Models
    • Asynchronous Programming Model (APM)
    • Event-based Asynchronous Patter (EAP)
    • Task-Based Asynchrony (TAP)
  • Demo – console app
    • AsyncContext.Run – can’t call async method from non-async method
    • AsyncContext – from Stephen Cleary – excellent book
    • This goes away in C# 7.1 – compiler will allow calling async method
    • Reading file
  • Misconception – that you create threads when you call async method
    • No, not true
    • Just because method is asynchronous, it won’t necessarily be on another thread
  • Use async when doing I/O bound stuff
    • Calling ReadLineAsync, you hit IO Completion Point; when done, let caller know
    • When I/O is done, calling thread can continue
    • Asynchronous calls may also not necessarily hit IOCP
  • If you do I/O in non-blocking way on a thread, you can then use thread to do CPU work while the I/O is happening
    • Performance really does matter–e.g. use fewer servers
  • When you call async method, compiler creates asynchronous state machine
    • Eric Lippert’s continuation blog post series
  • Task object has IsCompleted
    • Generated code need to first check state to see if task completed right away
  • Good news is–that you don’t need to write the asynch state machine plumbing code
  • Can use Tasks to run things in parallel
    • Task.Run(() => ..)
    • await Task.WhenAll(t1, t2);
  • Tasks are higher level abstraction than threads
  • Don’t ever do async void
    • Only use it for event handlers
  • Keep in mind that async tasks may actually run synchronously, under the covers


Demo – Evolving Expressions

  • Code uses all cores available


Locks

  • Don’t use them (or try not to use them)
  • If you get them wrong, you can get deadlocks
  • Don’t lock on
    • Strings – You could block other uses of string constant
    • Integer – you don’t lock on the same thing each time, since you lock on boxed integer
  • Just use object to lock on
  • Interlocked.Add/Decrement
    • For incrementing/decrementing integers
    • Faster
  • Tool recommendation: benchmark.net
  • SpinLock
    • Enter / Exit
    • Spins in a while loop
    • Can be slower than lock statement
    • Don’t use SpinLock unless you know it will be faster than other methods


Data Structures

  • List<T> is not thread-safe
  • ConcurrentStack<T>
    • Use TryPop
  • ImmutableStack<T>
    • When you modify, you must capture the new stack, result of the operation
    • Objects within the collection can change


Actors

  • Service Fabric Reliable Actors
  • Benefits
    • Resource isolation
    • Asynchronous communication, but single-threaded
      • The actor itself is single-threaded
      • So in writing the actor, you don’t need to worry about thread safety


Demo – Actors – Using Orleans

  • “Grains” for actors

TechEd NA 2014 – Async Best Practices for C# and VB

Posted by

1

TechEd North America 2014, Houston
Async Best Practices for C# and VB – Mads Torgersen

Day 4, 15 May 2014, 10:15AM-11:30AM (DEV-B362)

Disclaimer: This post contains my own thoughts and notes based on attending TechEd North America 2014 presentations. Some content maps directly to what was originally presented. Other content is paraphrased or represents my own thoughts and opinions and should not be construed as reflecting the opinion of either Microsoft, the presenters or the speakers.

Executive Summary—Sean’s takeaways

  • Avoid async void
  • Don’t use parallel threads for I/O bound code
    • Don’t want to waste threads in thread pool by having them waiting on I/O
  • Avoid event handler mess by using async/await
    • Use TaskCompletionSource to hook event to lambda that sets result of task
  • Don’t wrap synchronous code in async method
  • Don’t block UI thread to wait for completion of asynchronous code

Full video

Mads Torgersen– Program Manager, C# Language, Microsoft
One of the people responsible for the async feature

Key takeaways

  • Async void is only for top-level event handlers
  • Use threadpool for CPU-bound code, but not IO-bound
  • Use TaskCompletionSource to wrap Tasks around events
  • Libraries shouldn’t lie, and should be chunky

Async void is only for event handlers

  • User: if user clicks Print button too quickly, stuff not ready

Stop using async void

  • Unless you absolutely have to

Async void only for event handlers

  • Event handlers are async void
  • In your handler, you might call your own async void method
    • Then in 2nd method, you hit await
    • That method returns
  • Event handler then also awaits
  • ** slide – with arrows **
  • Can’t predict which order the methods will resume in

Variant #2 – Exception in GetResponseAsync

  • Exception doesn’t come back to original caller
  • async void method has no Task to put exception in
  • Then UI thread crashes, because no handler

How to fix

  • Make 2nd function Task return, then await it

Example 3 – virtual methods returning void

  • E.g. override of OnNavigatedTo, LoadState
  • The override calls base
  • But OnNavigated base calls overridden LoadState
  • Again, a race condition because we hit awaits and exit methods
  • Solution
    • Can’t return Task
    • Still hand off Task from caller to callee
    • Stick result of 2nd call in variable, after calling async Task helper method
    • Then in 1st method, can await on the variable!
    • Brilliant
  • Tasks always the best way to communicate completion

Example 4 – Can’t always see when you’re doing async void

  • Lambda
  • Lambdas may map to delegate that returns void
  • If both overloads offered, it picks Task-returning
  • E.g. If you Dispatcher.RunAsync with async lambda
    • When lambda returns, the caller thinks that work is done
  • Sol’n: find another way to communicate completion
    • Factor it out into async method that returns Task
  • Search for async (), check it

Async void only for event handlers

  • Principles
    • Fire-and-forget mechanism—almost never what you want
    • Caller unable to know when async void has finished
    • Caller unable to catch exceptions
  • Guidance
    • Use only for top-level event handlers
    • Use async Task-returning methods everywhere else
    • If you need fire-and-forget, be explicit
    • When you see async lambda, verify it

Threadpool

  • User: how should I parallelize my code
    • Loading list of housing data
    • Use Threadpool, task parallel library, parallel for?
  • Diagnose/Fix
    • Users code was not CPU-bound

Threadpool – sequential

  • Sequentially, you have to wait

Threadpool – Try #1

  • Parallel.For
    • Deserialize and add to list
    • Runs on parallel cores
  • Now down to 300ms, from 500ms
  • But is it really taking 100 ms per house?
    • Nope, it’s actually I/O bound
    • Deserialize is actually blocking on I/O

Is it CPU-bound or I/O-bound?

  • CPU-bound – should parallelize
  • I/O-bound – maybe not

How it works

  • Doing two threads, two cores
  • Gradually spins up threads, as it sees first thread waiting on I/O
  • So we created more threads than we need

How to code it right

  • Parallelize I/O bound code
  • List of tasks
  • LoadFromDatabaseAsync
  • Then: await Task.WhenAll(tasks)
  • No threadpool
  • Thread should not be waiting on I/O

Threadpool – may get another bottleneck?

  • Moving off threadpool, but doing I/O in parallel may lead to I/O bottleneck if you have a large number of tasks
  • So use a queue and workers
  • Use WorkerAsync
  • Create three workers

Threadpool

  • Principles
    • CPU-bound okay on threads
    • Parallel.ForEach and Task.Run are good way to put CPU-bound work onto thread pool
    • E.g. LINQ-over-objects, computational
    • Use of threads won’t increase throughput on machine that’s under load
  • Guidelines
    • For IO-bound work, use await, rather than background threads
    • For CPU-bound work, use background threads via Parallel.ForEach or Task.Run, unless you’re writing library or scalable server-side code

Async over events

  • User
    • UI code looks like spaghetti
  • Diagnose/Fix
    • Events are the problem
    • Consider wrapping them as Tasks

Apple picking game

  • Multiple levels of events
  • Sequence of things all listed as nested lambdas
  • Ick !
  • Callback hell
  • Solution
    • State machine
    • Becomes complicated in a new way
  • Now we have everything as global events

The problem is events—they’re not going away

Solution

  • await async events
  • Trick – how to turn helper methods into async

Async over events – async helper methods

  • Use TaskCompletionSource<object>
  • Guy who creates TCS controls how task completes
  • Make your own task, rather than letting async create a task for you
  • Lambda just tells task that things are done
  • Then wire this lambda into Completed event
  • Then you await this tcs.Task
  • So—it’s about converted synchronous work with Completed handler into Task-based paradigm
  • Fantastic!

Async keyword is for creating logic around methods that are already async

  • When you want to create your own Task, use TaskCompletionSource

Wow, this is great. Learning async tricks from Mads..

Async over events

  • Principles
    • Callback-based programming, as with events, is hard
  • Guidance
    • If event handlers are largely independent, leave them as events
    • If they look like state machine, then await is maybe easier
    • To turn event into awaitable tasks, use TaskCompletionSource

Library methods shouldn’t lie

  • Signature hints at whether a method is synchronous or asynchronous

Library methods shouldn’t lie

  • Honest about synchronous
    • Some methods do actual work, occupy work
    • You should say so, synchronous
  • Synchronous methods
    • Do work
    • You’re not wasting your time waiting for me
  • Asynchronous methods
    • I’ll initiate something, but return immediately

Library methods shouldn’t lie – Example

  • Synchronous
    • PausePrint – burns CPU
  • Asynchronous
    • PausePrintAsync – await Task.Delay(10000)
    • Honest, because it returns immediately, spawns task
  • Don’t: wrap synchronous code in async method
    • Returns Task.Run (return it)
    • This method lies—it’s not really async
  • Never: wrap asynchronous code in synchronous
    • Async method that returns synchronously
    • Synchronous wrappers for asynchronous work – NO !

Dangers of wrapping synchronous in asynchronous method

  • Wrap synchronous in async
    • You’re still doing work
    • Hiding from app dev where work is being done
  • Threadpool is app-global resource
    • Scalability hurt
  • On server, spinning up threads hurts scalability
  • App is the best position to manage its threads
    • Don’t use threads in secret

Dangers of blocking – wrap asynchronous in synchronous

  • LoadAsync
  • Then wait on this in button click handler
  • And then update view
  • Rather than doing await so that handler returns immediately
  • LoadAsync works fine—creates thread
  • Blocks UI thread—bad !
  • Then LoadAsync does await and it leads to deadlock
  • Because the resumption of LoadAsync, after await, wants UI thread
  • But you’ve blocked UI thread—crap
  • A bit better in thread pool

Library methods shouldn’t lie

  • Principles
    • Threadpool is app-global resource
    • Poor use of threadpool hurts scalability
  • Guidance
    • Help callers understand how your method behaves
    • Libraries shouldn’t use threadpool in secret
    • Use async signature only for truly async methods

Async – not spinning on threads

Sync – not blocking threads

Libraries should expose chunky async APIs

  • We all know sync methods are “cheap”
    • Years of optimizations around sync methods
    • Enables refactoring at will
  • E.g. synchronous string out
    • IL is simple
  • Async method that outputs string (but doesn’t wait)
    • Body of method is 3x longer
    • Has to initialize state machine
    • Lots of plumbing
  • Important mental model
    • How many allocations are required for async state machine?
    • Allocation will eventually require GC
    • Garbage collection is what’s costly

Fast Path in awaits

  • Each async method has to allocate
    • State machine class holding method’s local variables
    • Delegate
    • Returned Task object
  • If code path doesn’t hit any awaits
    • Optimized so that state machine and delegate aren’t allocated. Just Task
  • If awaited Task has already completed, then skip actual wait
  • If you don’t have any awaits fired and you have common result (e.g. 0, 1, true, false, null, “”)
    • Compiler just grabs these pre-gen’d task and just returns it
  • You can follow this same pattern, for common return values in Tasks
    • Create wrapper

Libraries should expose chunky async APIs

  • Principles
    • Heap is an app-global resource
  • Guidance
    • Libraries should expose chunky async APIs, not chatty
    • If library has to be chatty, and GC perf is problem, and heap has lots of async allocations
      • Then optimize the fast-path
    • Generally, use async to your heart’s content and don’t worry about it
    • But just be aware of what’s going on under the hood

Consider .ConfigureAwait(false) in libraries

  • Sync context represents a “target for work”
    • E.g. DispatcherSynchronizationContext, whose .Post() does Dispatcher.BeginInvoke()
    • Sort of “where do I live”
    • When await resumes execution, it has to look at sync context to figure out what thread to run code on. E.g. On UI thread
    • Goal: after await, you should be where you were before (e.g. I’m still on the UI thread)
    • Extra level of bookkeeping can be expensive
  • Library code often doesn’t care where it’s running
    • You can ask await to not go find original context after await, but just keep running in whatever context is current
  • “Await task” uses the sync context
    • await task.ConfigureAwait(false)
    • Suppresses SyncContext.Post()

Consider .ConfigureAwait(false)

  • Principles
    • UI message-queue is app-global resource
    • Too much use will hurt UI responsiveness
  • Guidance
    • If you call chatty async APIs but doesn’t touch the UI, use ConfigureAwait(false)

Resources for async Best Practices