Tuesday, August 31, 2010

DEV305 Parallelize Your Microsoft .NET Framework-Based Applications with Parallel Extensions

Speaker: Ivan Towlson

Tech.Ed Online

Take advantage or newer processor architecture.

.NET Framework 4 has inbuilt functionality for using this new functionality.

Like LINQ, but faster.

Querying a large dataset.

Traditional approach requires large number of considerations for treading.



ParExtSamples

Works for anu IEnumerable
- Optimizations for other tyoes(T[], IList)

Writing a PLINQ query
- .AsParallel()
- ParallelEnumerable (Does the actual work)

Partitioning
- Operators are replicated across partitions for (ideally) complete isolation

Operator Fusion
- Avoids excessive merging and partitioning steps

Partitioning Algorithms
- Chunk - potential for a lot of locking when doing small tasks
  Progressively hands out more work out to each thread to separate both small and large jobs.
- Range - For IList equally divide work among threads
- Stripe - For IList
  Elements handed out round-robin to each other
  Less wasted effort in the case of exception
- Hash - For IEnumerable
  Elements assined to partition based on hash code.
- Custom - Partitioner

Merging
- Pipelined: Each result becomes available as they are completed.
- Stop-and-go: For OrderBy, ToArray, ToList, etc...
  Higher latency and more memory usage.
- Inverted: no merging needed
  ForAll extension method - no buffering required.

Parallelism Blockers
- Ordering not guaranteed (consider OrderBy)
- Exceptions (Stop other threads?) - System.AggregateException
- Thread affinity - Web form controls
- Operations with < 1.0 speedup - Overhead can make the entire operation slower.
- Side effects and mutability are serious issues
  Avoid shared state, side effects  

Task Parallel Library (TPL)
- Parallel.For(o, N, i => { work(i);});
- Parallel.ForEach

Regions
- Parallel.Invoke();

Task Concepts
- FromAync

Coordination Data Structures
- Thread-safe scalable collections
- Task and data management
- Synchronisation types
    SpinLock and SpinWait for very quick waiting periods.

DEV202 Pimp My App

Speaker: Shane Morris @shanemo

- lack of whitespace causes clutter - very flat
- colour balance across the page drawing the eye to one location.
- promote symmetry.
- where is the logical starting point in the page. Queues.
- What do we want people to see and in what order?
- Uppercase is harder to read due to outline versus mixed case text.
- What is lower down on the page?
- Avoid spilt navigation

1. Graphics come last, not first.
2. Set goals
- User, business, usability, experience.
3. "You are not your user
4. test early and often
5. Go with the flow 
Start with navigation and flow of the tasks


Layout steps
1. Map out the workflow
- What would a first time user do.
2. List your content
- Widgets
- quince.infragistics.com Pattern Library
3. Layout the elements in order
- left-right, top-bottom.
4. Check natural grouings
- Keep releated concepts together.

Presentation
1. Remove every unnecessary element.
2. Minimise Variation
- Err on the side of consistency.
3. Colours 
"Red and green should not be seen without a colour in between."
"Angry fruit salad."
Rate 1 to 5 for visual design skills and use that many colours.
Neutral background colours are safest. (white, grey, black, navy blue,)

Pull colours from a suitable photo.

kuler.adobe.com
colourlovers.com

About 10% of males have some form of colour blindness. Vischeck.com

How many diffent type faces shoul I use?

3. Line stuff up.
- Beware unintended relationships

4. Space and Size things evenly

5. Indicate Grouping
- Group boxes
- Similirity
- Proximity
- Alignment
- Empty Space

 Visual Hierarchy

6. Adjust Visual weight
Hot colours,
Size,
Contrast
Irregular Shapes
Misalignment
3D
Animation.

DEV208 Getting Started with Workflow in .NET 4

Speakers: Stefan Sewell, Peter Goodman

What is a workflow?
- Visual sequence of connected steps. (Flowchart or state machine)
- Declarative (what we want, not how to achieve it) - SQL, HTML
- Runtime schedules execution of the steps.
- activities linked together with data that flows though and is executed by the runtume.

Why would I use a workflow?
- Business processes that can naturally map to a workflow
- Very good at handling async work. Runtime to handle the scheduling.
- Long running logical processes that are episodic in nature. Persist/Restore. Human based workflow in minutes/hours/days.
- intuitive customisation model as part of your solution. Visual DSL.

How do I create a workflow?
-  Use the System.Activities namespace rather than the older System.Activities namespace.
public InArgument Question {get; set;}
InArgument ties the properties to the runtime instance of the workflow.

"That would be an ecumenical matter."

Can write unit tests for Workflow activities.

Workflow Basics

Further reading
Total Noobs' 
SOA206


How do I execute a workflow?

Workflow Service

A WCF service whose implementation is a workflow.

Service logic easily modelled as a workflow.

COS304 Azure Storage Deep Dive

Speaker: Chris Auld

Scalable - You will run out of money before disk.

Exposed

15 cents per gigabyte to store.

RESTful Web Services
- Use Azure or local apps.

Asian data centres are more expensive for data.
.
Can CDN Enable Account

Account is secured with a 512 bit shared secret key.
500 TB per account.

Storage in the Development Fabric
 - good for developing offline.
 - costs minimal (cents) if developing with less than 1 GB of data a month.

Storage Security

- HTTPS - Digitally sign requests for privileged operations.

Azure Storage Abstractions

Blobs - Could be used to serve static content. 60 MB/s output speed. Account Container - Grouping of blobs. Limited throughput per container. Blob - Identified by name Pages / Blocks - two types of blob. Blob is always accessed by name. Special $root container. - Allows for definition of clientaccesspolicy.xml http://[account].blob.core.windows.net/[containter]/[blobname] Can use prefix/delimiter and special blob naming to simulate directory structure. Pagination - returns continuation token (MarkerValue) to continue beyond page. Use Affinity group to keep computing and storage together and avoid paying for extra network traffic. Block Blob - used for streaming an entire file. ETags - versioning support over HTML Block blobs - can be uploaded in parallel via blocks and then recombined in Windows Azure Storage. Makes retry more efficient. Page Blob - targeted at random read/write workloads. Fixed 512 bytes Shared Access Signatures - E.g. Grant read access to a certain blob for a period of time and then pass to client as a URL. Revoke by timeout or via a container level policy that can be deleted. Ad Hoc Signatures. Policy Based Signature - container level policy allows revoking. Drives - Wrap the blob storage and allow NTFS volumes Tables - Queues -

Content Delivery Network (CDN)

Public blobs can be served via the Windows Azure CDN URL.
Could map custom domain to the CDN.

INO301 Code Different

Speaker: Ivan Towlson

C# and Visual Basic reconsidered
- Imperative
- Textual
- No extensibility mechanisms
- Minimal abstraction over the underlying machine

The assembly languages of the CLR

F#
- A functional-object hybrid
- Included in Visual Studio 2010
- A CLR language

You can call C# or Visual Basic components from F#
You can call F# components from C# or Visual Basic

F# Features With C#/VB Equivalents

First class functions
Similar to .NET delegates 

Lambdas and closures
Similar to C# lambdas Type inference

Similar to C# var, but used much more extensively
You rarely have to write explicit type annotations in F#

Comprehensions
Similar to LINQ

Quotations
Similar to expression trees

F# Idioms

Emphasis on functions rather than records

Rich data type support: options, tuples, discriminated unions
Pattern matching

Immutable by default

Helps with asynchronous and parallel programming
“Expressions that have values” instead of “statements that perform actions”

F# Interactive REPL
Try out your code as you go (tip: Alt+Enter!)

F# Features

Asynchronous workflows and agents

Example: parallel or cloud computing interop Starcraft

Computation expressions

Customising how a sequence of expressions is executed

Parser generator

Example: simple macro or expression languages

Units of measure

Example: scientific or financial computation

WEB302 Bringing the Web to Life with jQuery

Speaker: Jeremy Boyd

Visual Studio Intellisense support.
Works well with MVC and the existing MS Ajax libraries

Helps avoid cross browser issues.

Microsoft and Google CDN versions

Requires good HTML structure for selectors.

doc.jquery.com

xval for doing validation between client and server.

1. Expanders
- Expanding and collapsing divs.
$("selector").hide();
$("selector").fadeIn();
$("selector").slideToggle();

Select sets using CSS3 selectors.

2. Buttons
- $("#go").click(function() { $("form").submit(); return false; } );

3. Highlighting Errored Form Elements
- .css('prop'. 'val');
.addClass('foo');

$(":input").removeClass("errored");

4. Consuming server side JSON data
Autocomplete textbox.
$.post(url, payload, callback);
$.ajax({ type: 'POST', ...});

FlexBox

JsonResult

5. jQueryui
http://jqueryui.com
UI library of widgets, animation

.accordion()

6. 3rd Party Plugins

7. LightBox
FancyBox

Easing - function over time for animation.

8. Writing your own plugins
- plugin authoring

.html()
.live() - for anything that matches the selector now or in the future apply this selector.

9. Copying a cool technique you have seen


Monday, August 30, 2010

DEV304 Increase Productivity and Reduce Costs with Visual Studio Lab Management

Speaker: Anthony Borton

Tech.Ed Online

Automate virtual machine provisioning
Checkpoint as needed
Test and debug multiple environments
Requires
* Team Foundation Server 2010
* Microsoft Test Manager 2010

Includes - Microsoft Test Manager 2010

A Logical Configuration
Active Directory, SCVMM, VM Library (Template library), Hyper-V Host, TFS & Lab Management, Build Server, Developer Machines, Tester Machines.

SEC302 Hack-Ed II: Stop the hacking

Presenters: Andy Prow, Kirk Jackson

Tech.Ed Online

oWasp.org

Enumerating usernames - finding valid users

Leaking content out to Facebook

Threat modelling

Counter-measures

Text file with HTML in IE can run as HTML.
X-Content-Type-Options nosniff
Don't sniff any HTML content out of files.

Serve content via a handler rather than the web servers file system. E.g. They could upload a ASPX file to the upload folder and potentially run it.
Omit Content-Disposition: attachment; filename=%lt;file>

Tab Nabbing
- Change the Title
- Change the favicon
- Load the target site over the top of harvesting site.

DEV207 What's new in C# and VB.NET 4.0

Speaker: Ivan T

Tech.Ed Online

C# 4.0

Generic co- and contra-variance

Covariance
Contravariance

IEnumerable<t> is safe as only enumerating members. Won't be adding other subtype to collection.

IComparer<t>
-

If T appears onlay as an output, it's safe to pass X<tderived> for X<t>
- covariance
- C# syntax X<out>

If T appears only as an input, it's safe to pass X for X
- countravariance
- C# syntax X<int>

List and IList are not supported but IComparer and IEnumerable do support.

Optional and named parameters
- Optional
public static virtual DoStuff(int first, int second = 123, int third = 456)
{
}

DoStuff(888);

Not creating an Overload.
Default value is stored in the calling assembly. Something to consider when versioning the libraries. Not for external publicly visible methods.

DoStuff(888, third: 999);

Dynamic Programming
Dynamic Language Runtime

COM interop
"No PIA" Primary Interop Assembly
- Merge COM interop type definitions into the calling assembly.
- Reduces total project size.

COM interop  enhancements
- Can now leave out unnecessary arguments, auto fill in "ref missing" arguments and identify parameters by name.
- Dynamic helps with weak typing problem.

Tuple
- Simple way of packaging up data into a single object without creating the actual implementation. Not meaningful property names.
Tuple.Create(true, "bob");
Imutable

WEB202: Designing for HTML5 and CSS3

Speaker: Darren Wood @darren
http://www.dontcom.com
http://www.slideshare.com/darren131

TechEd Online

WHATWG - HTML5 workgroup. Different to the W3C
Ian Hickson

Why HTML 5?
- Support existing content
- Much reduced HEAD. E.g. Don't need to specify JavaScript as the scripting language used in the page.
- Less bytes, being more mobile friendly.

New Elements
article, aside, audio, canvas, command, datalist, details,figure, footer, header, hgroup, keygen, video.

header
- Replaces common CSS class that appears on the majority of pages.

footer
- E.g. contains copyright notice.

nav
Links to other documents. Primary navigation.

aside
- Related content. Separate to the main content but related.

section
- A section of a document. Can be nested.
E.g chapters of a book. Each section can contain an H1.'

article
- Similar to a section.
Use an article if it could be pulled out of the page and still be meaningful.

video
- Native video support.
- Can have a default image.
- Can provide different content for different browser.
audio
- Very similar to video tag.

Forms.
New input types.
- search
- email
- url
- tel
- range
- number
- date
- datetime
- datetime-local
- time

Attributes
- placeholder = "enter something here"
- autofocus
- Required
- autocomplete="off"

Semantics
- Less divs and meaningless markup.

Mircoformats.org
- Add further information.

Can now wrap block level elements in a anchor a

Javascript

Drag & Drop

Canvas
- Dynamic images.

document.getElementsByClassName('test');
document.querySelectorAll('.testClass');

localStorage
 - Megabytes of local storage with query syntax

Offline Application Cache.
- Good for mobile web apps

CSS3
Selectors
- Substring selector (begins, ends, contains)

Pseudo-class
- Match odd li elements
- li:last child.

p:empty
p::selected

Properties

- Vendor Prefixes
- opacity
- rgba - colour including alpha

Multiple background images to one element. Position them separately.

Image borders. border-image.

border-radius:
border-top-right-radius:

text-overflow: ellipse;

Transforms
- Good for roll overs
- skew, scale, rotate
- fade in and out on hover.

@font-face
http://typekit.com
http://www.fontsquirrel.com
https://fonts.google.com/

http://modernizr.com - helps older browsers work with HTML 5. Allows for CSS selectors for determining support level. Also object on the DOM for checking support.

http://mediaelementjs.com - script for embedding video and audio
http://css3pie.com - 
http://css3please.com - Shows cross browser versions of markup.
http://www.html5test.com - tells HTML support features for current browser.
http://www.html5rocks.com - Examples of what is capable.
http://html5doctor.com - Write ups on new elements, problems, issues.
http://goo.gl/51ma HTML5 schema for Visual Studio 2010.

FDN001 From Concept To Reality with Developer Platform and Tools

Speaker(s): Ryan Tarak

  • TFS Power Tools
  • Visualisation and Modeling Feature Pack - MSDN subscribers only
  • Generate Dependency Graph - dgml file type
  • UML Modeling support and doge generation
  • Productivity Power Tools
  • Windows Server AppFabric
  • Deploy, Scale, Manage
  • AppFabric Dashboard

Cloud

  • Low capital investment with the ability to expand quickly.
  • Windows Azure Tools for Visual Studio 1.2
  • Package, deploy and debug support.
  • Client, Browser, Phone
  • WCF RIA Services
  • Rich Internet Application Services - support drag and drop for silverlight.
  • Expression 4 - single tool for targeting desktop/phone/web
  • SketchFlow
  • In Expression 4 for prototyping applications
  • Easily translates towards final product
  • TFS Integration for customer feedback.
  • DEV306 WEB307 WPH203

Notes from MS Communities Code Camp in Auckland

Security - 10 things YOU are doing wrong! - Kirk Jackson

- Rainbow table - used for reverse hash lookup
- RNGCryptoServiceProvider is the default implementation of a security standards compliant random number generator. If you need a random variable for security purposes, you must use this class, or an equivalent, but don't use System.Random because it is highly predictable.

- email as username - verify that they own the email address in question.
- autocomplete=off on inputs such as Credit card number to avoid storing the data locally. Also check PCI compliance.
- invalidate session on login.
- Presentation using prezi.com

Windows Phone 7 Introduction

5 Languages in 50 minutes

- iolanguage
* prototype language similar to java script
* Uppercase indicates types. Lowercase instance.
* Zero index lists
* true is a singleton
* DSL Domain Specific Language
* Actors for concurrency

-Prolog
* Logic rules based language.

- Scala
* Runs on JVM
* Similar to F#
* values are immutable

- Erlang
* Ericsson Language
* Allows for crashing and resuming

- Clojure
* Lisp on the JVM

An introduction to artificial intelligence techniques

- Constraint satisfaction
Scheduling: time boxes and dependencies

Genetic algorithms
Fitness function
Shakespearean Monkeys

Deduction and reasoning
- Uncertain Reasoning
- Bayesian reasoning

Support vector machine - divide up the data space
SVM.NET

Learning = feedback
supervised learning - training data
Neural Network connected up in layers - input nodes and output nodes

Reinforcement learning - rewarded or punished on the basis of the outcome.
Unsupervised learning

Natural language processing
Grammar
F# Parser generator

Image analysis
Edge recognition
Segmentation

Motion detection

Recognition

The Microsoft Application Server: What is it ...

Windows Server AppFabric - @ryancrawcour
What is the problem?
A: Who should build application infrastructure? plumbing
Microsofts application infrastructure - pipes etc...

Ships as part of Windows Server (soon) - switch on like IIS

Two distinct components
* Caching (codename Velocity)
* Hosting for WCF and WF (codename Dublin)

AppFabric Caching Services
What is the problem?
Distributed caching mechanism.
Improves speed, scalability and availiablity of applications that use data?
Large improvement to applications that primarily retrieve data.
* Cache in memory on single computer. Doesn't work well with load balancing.
* Could throw more hardware at the database.
* Distribute your cache on to shared computers. (AppFabric Caching Services provides this)

Cache Cluster - machines within the cluster hidden from the application using the cluster

AppFabric Hosting Service
What's is the problem?
* Build your own host
* Using IIS(http protocol only)/WAS (on top of IIS for extending bindings) hosting as is
 Difficult to monitor service state
* " with extended capabilities & management

WCF - foundation for building & consuming services
Client connects to endpoints over protocol

WF Windows workflow foundation
Model workflow activities of business process.

Workflow Services
Wrap Workflow in a WCF service. This is very similar to what BizTalk does.
Why implement the service as a workflow?
Not for database calls.
Business application - E.g. take on new staff. Setup PC, username, parking space, payroll.

AppFabric Hosting.
IIS Worker Process

Workflow is good for long running processes.

Persistence Store.

Monitoring Database. AppFabric and monitor events.

AppFabric Extensions to IIS Manager for monitoring status of Workflow and WCF.
- AppFabric DashBoard

AppFabric & Biztalk
* AppFabrix hosting services is for hosting applications
* Biztalk is for integrating systems.

Windows Azure AppFabric
* Service Bus
* Access Control

Think Async LINQ with the .NET Reactive Extensions

SuperCoder 2000

LINQ & RX (Orion Edwards - @borland)
Async - write a lot more code and tends to be harder to read intention that blocking code.
Reactive Framework - Rx
Google Reactive extensions
Add as reference once installed to project.

LINQ Fundamentals
- Any collection or stream is just a series of values
- Can be lazy loaded - values over time
- IEnumerable provides a unifeid interface to pull values.

Rx Fundamentals
- Async operations are also a sequence of values over time, but the values are pushed.
- Rx provides IObservable, a unified interface for any source to push over time.
- IObservable is the mathematical dual of IEnumberable
Use standard LINQ querying.


IObservable for PUSH based sequences
- Subscribe
EnumerateFilesAsync

Unified for
Push IEnumerable
Pull IObservable

Concurrency
IEnumberable on your thread.
IObservable will push on its thread. I.e. Mouse events are on the dispatcher thread.
Redirect to another thread. Dispatcher.BeginInvoke,

IScheduler
.ObserveOn to pass to scheduler.

Wrap up begin end pairs to make IObservable

ObserviceOnDispatcher()

Tuesday, August 24, 2010

Disabling and/or adjusting the drag and drop activation distance

Updated: A better Visual Studio 2010 specific solution is to install the VSCommands 2010 plugin.

Several times I've caught myself out in Visual Studio Solution Explorer by dragging and dropping a file/folder into another folder by accident. Usually the folder immediately adjacent to the file/folder being moved. The default windows threshold for drag and drop it 4 pixels. By increasing this I'm hoping to avoid accidental drag and drop.

Note: This will affect all drag and drop operations in Windows, not just Visual Studio.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;

namespace AdjustDragDropDistance
{
 class Program
 {
  /// 
  /// uAction for drag and drop horizontal threshold
  /// 
  static int SPI_SETDRAGWIDTH = 76;
  /// 
  /// uAction for drag and drop vertical threshold
  /// 
  static int SPI_SETDRAGHEIGHT = 77;
  [DllImport("user32.dll", CharSet = CharSet.Auto)]
  static extern int SystemParametersInfo(int uAction, int uParam, string lpvParam, int fuWinIni);

  static void Main(string[] args)
  {
   //Default value to use the can be overridden from the command line.
   //The normal windows default is around 4 pixels.
   int _dragThresholdInPixels = 25;

   try
   {
    if (args.Length > 0)
    {
     if(!int.TryParse(args[0], out _dragThresholdInPixels))
     {
      Console.Error.WriteLine("Failed to parse [{0}] as integer for drag threshold.", args[0]);
     }
    }
    SystemParametersInfo(SPI_SETDRAGWIDTH, _dragThresholdInPixels, "", 0);
    SystemParametersInfo(SPI_SETDRAGHEIGHT, _dragThresholdInPixels, "", 0);
   }
   catch (Exception ex)
   {
    Console.Error.WriteLine(ex.ToString());
    Console.Read();
   }
  }
 }
}

See Also:

Monday, August 23, 2010

Inserting file data into a varbinary(max) column

The following are ways I've found to import binary file data into a VARBINARY column on SQL Server 2005.

OPENROWSET BULK

This method avoids the need for a front end tool such as SSIS.

CREATE TABLE documentTable(Document varbinary(max)) 

INSERT INTO documentTable(Document) 
SELECT * FROM 
OPENROWSET(BULK N'C:\ImageFile.png', SINGLE_BLOB) as imagedata

Note: The file path is with respect to the SQL Server. I.e. it is loading a image file from the servers file system.

See Also:

SSIS

If SSIS is available another option was to use a SSIS package as per SSIS – Importing Binary Files Into A VARBINARY(MAX) Column.

Scripting out to XML with base64 encoding

SELECT * FROM [documentTable] for xml raw, BINARY BASE64

DECLARE @xmlDoc xml
SET @xmlDoc = 
'

'

This data can then be scripted in.

DECLARE @xmlDoc xml
SET @xmlDoc = 
'

'

DECLARE @idoc int

EXEC sp_xml_preparedocument @idoc OUTPUT, @xmlDoc

INSERT INTO [myTable] ([Document])
SELECT [udfBase64Decode]([Document]) as [Document] FROM openxml(@idoc, '/ROOT/row') 
 WITH (
  
  [Document] varchar(max) '@Document'
  )

EXEC sp_xml_removedocument @idoc