Lab Management with TFS 2010
8 likes2,886 views
The document discusses the features and improvements of Visual Studio 2010 and Team Foundation Server (TFS) for enhancing quality assurance (QA) processes, including streamlined test management and developer-tester collaboration. It addresses common frustrations in development teams, outlines goals for tighter integration of QA with the development lifecycle, and introduces various testing capabilities such as coded UI testing and automated test case management. Additionally, it highlights the role of virtual lab management in simplifying environment setup and improving overall testing efficiency.
![[Custom Data Collector]35Bug Form auto-filled with step results & step attachments](https://image.slidesharecdn.com/labmanagementwithtfs-110112035222-phpapp02/85/Lab-Management-with-TFS-2010-62-320.jpg)
Lab Management with TFS 2010
- 1. Virtual Lab ManagementTeam Foundation Server 2010Ed BlankenshipMicrosoft MVP (TFS, Visual Studio ALM, and VSTS)Notion Solutions, Inc.www.notionsolutions.com
- 2. About Ed BlankenshipMicrosoft MVP of the Year
- 3. ALM/TFS Consultant at Notion Solutions
- 4. Live in Charleston, SC
- 8. Author - Wrox TFS 2010 Book: http://bit.ly/TFS2010BookFrustrations in development teams“my testers are spending too long testing the same thing”“tooling is expensive (time, licenses, people)”“the developers say the defects are useless”“when is my software ready to ship?”“we have a changed requirement, what do I need to test?”“developers and testers work in silos and don’t communicate/speak the same language”3
- 9. Customer Challenges QA is not aligned with lifecycleDevelopers and testers work in silosQA is not leveraged
- 10. Goals for 2010 Release - TestingAlign QA with the LifecycleCreate tighter Dev/Test interactionEnable highly leveraged QA teams
- 11. Visual Studio 2010 Test Capabilitiesspecialistgeneralistcoded ui testunit testingweb performance testtest runnerload testtest case managementvirtual lab managementdata diagnostic adapters (video, action log, event log etc)team foundation server with reporting(bugs, requirements, user stories, source control, build)
- 13. Test PlanningTest ProcessRequirementRequirementTest WritingRequirementTest AutomationTest ExecutionReport Results
- 14. Testing FeaturesQA Professional focused interface
- 15. Work Item based test case management, test planning and status tracking
- 17. Identification of high-priority tests
- 18. Robust manual testing functionality
- 19. Test definition
- 20. User-friendly Manual Test Runner
- 21. Defect data capture tools
- 22. Automated user interface testingVisual Studio & TFS Testing ToolsTest Planning
- 25. Test Recording
- 26. Data Collectors
- 29. Load Testing
- 30. Generic Tests
- 31. Ordered Tests
- 32. Unit Tests11Test Case Management
- 33. 12Test Case is first class work item in TFSDetailed steps along with expected results
- 34. 13Data drive a test caseSpecify data in parameter table
- 35. 14Select common steps and convert to Shared Steps
- 36. 15Shared Steps allows for common steps to be reused across test cases
- 37. 16Organize Tests using Static (folder) suite
- 38. 17Or create new test cases & add to suiteAdd existing Test cases to suite
- 39. 18Organize Tests using Requirements based suite
- 40. 19Select requirement and add to plan
- 41. 20Tests already linked to requirement are added to plan
- 42. 21Assign tests to team members
- 43. 22Organize Tests using Query Based (dynamic) suite
- 45. 24Tests matching query are added to suite
- 46. 25Rich Bug and Test Runner
- 47. Better Tester / Developer CollaborationAutomatically gathers defect and runtime details
- 48. Eliminates the “ping pong” effect
- 49. Eliminates the “No Repro” problem27Select Test Case & click Run
- 50. 28Microsoft Test Runner starts up
- 51. 29Mark steps (pass/fail) during testingError free data entry of parameters using CTRL+V
- 52. 30Mark step as fail – bug found
- 53. 31Add comment
- 55. 33Click on Create Bug
- 58. Event Logs
- 59. IntelliTrace
- 60. Test Impact
- 61. Video Recorder
- 62. [Custom Data Collector]35Bug Form auto-filled with step results & step attachments
- 64. 37Data collector logs auto attached
- 65. 38Exact actions performed for each step
- 66. 39System Info across all VMs in environment
- 67. 40Link to test case auto-created
- 68. 41Just enter Title and save the bug
- 70. Start Debug
- 71. Encounter defect
- 73. Move back through execution history to locate defect
- 74. Historical debug trace enables debugging offline test runs
- 75. “As you sit in the debugger, being able to move backwards while in the debugger (or attach the execution log to a bug) has to be the killer feature coming up in the entire Visual Studio 2010 release.” -John RobbinsIntelliTrace
- 76. Test Impact AnalysisIdentifies tests impacted by code changes
- 77. Significantly reduces test timeCoded UI Testing
- 78. Coded UI TestingEnables verification of UI functionality.Record test actions and VS will generate the resulting code.Assertions can be added to verify functionality.Like all VS tests, results can be stored in TFS.
- 79. 47From Test->new test, click Coded UI test, specify test name and project name
- 80. 48Use existing action recording to generate code
- 81. 49Specify test case which has action recording and click OK
- 82. 50Code generated from action recording – datasource as TestCase
- 83. 51Code generated handles data parameterization
- 84. 52Invoke Coded UI Test Builder for adding assertions
- 85. 53Drag cross-hair onto the control you want to add assertions on
- 86. 54Properties of selected control and its UIMap is shown
- 87. 55Select property to assert on and right click to add assertion
- 88. 56Add the assert
- 89. 57Specify Assert method name and generate code
- 90. 58Code generated for assert method
- 91. 59Add solution to source control
- 93. 61Go to test case that you want to associate the newly created automation
- 94. 62Specify test method and click OK; save test case
- 95. 63Newly automated test case will be picked up by dynamic query based suite and run as part of build-deploy-test workflow
- 98. Test Lab BenefitsSimplifies environment setup
- 99. Simplifies testing multiple configurations
- 100. Enables reverting to baseline configuration
- 101. Provides checkpoints to aid defect resolution
- 102. Improves test hardware utilizationLab Management GoalsVirtualize test lab resources
- 103. Lower technical requirements for test environment creation
- 104. Enable environment recreation for defect resolutionTeam Foundation Server (TFS)Hyper-V HostsLibrary SharesLab Management in Visual Studio 2010LS1VM1LS2VM2System Central Virtual Machine Manager (SCVMM)Lab ManagementLSnVMnTest Case ManagementAuomated BuildsWork Item TrackingVersion Control
- 105. Template TypesVirtual Machine Template
- 106. One “golden” VM in a specific configuration
- 107. i.e. Win 7 x64, with SQL 2008 R2 & Office
- 108. Environment Template
- 109. Multiple VM’s in a specific combination70Create Virtual Environment from Lab Center
- 110. 71Specify name and location for environment
- 111. 72Pick role for VMSelect VM from template library
- 112. 73Specify capabilities – “Run Tests” ; “Run Workflows” ; “Network Isolation”
- 113. 74Environment (collection of VMs) created
- 114. 75Manage snapshots for environmentAccess environment using Lab Environment Viewer
- 116. 77Dev clicks on lvr link & selects connect to snapshot of the environment
- 117. 78Dev is able to access the snapshot’ed environment via LE viewer
- 118. Lab Management 2010 Capability/FeaturesVirtual Environments (Hyper-V)Physical environmentsVirtual environments (non Hyper-V e.g. VMWare)Visual Studio 2010 Capability/FeatureTestingRunning unit testsRunning manual testsRunning automated, load, and coded UI testsFiling rich bugs with diagnostic adaptersBuild deploymentAutomated build-deploy-test workflow Environment creation and managementCreate environment from VM templatesStart/stop/snapshot environmentConnect to environment using Environment ViewerClone environments using network isolationSupported out of boxNot supported, but possible with customizationNot Supported N/A
- 119. 80Automated Build – Deploy - Test
- 120. 81Create new build workflow
- 121. 82Provide a name
- 122. 83Specify Build Controller and output drop folder
- 125. 86Select environmentRevert to a specific snapshot
- 126. 87Select build definition to build the application
- 127. 88Specify deploy scriptsTake snapshot after deployment
- 128. 89Specify tests to run post deployment
- 131. By Requirement
- 132. By Automated Build
- 133. Quality Dashboard
- 134. Bug Tracking ReportsImproved Quality Reporting
- 140. DashboardsExcel Services ReportsTeam Web AccessSharepoint Web Parts
- 141. Questions?
Editor's Notes
- #5: We found that customers pain points bucketed themselves into three things areasTesters are not aligned with the lifecycle. Finding out a code change is impacting their day at work (test more, go home late, miss the time with kids). Testers and developers work in silo’s often causing significant dis-satisfaction and un-happiness and creating an environment of lack of respect across the two teams. Testers are smart, developers often don’t give them the respect they deserve because they see testers filing poor defects, not really impacting quality. Testing teams are not really leveraged well as they waste so much time doing the basic things. Just getting their bugs looked at take so much time that they cannot spend quality time doing what they love and provide better service to the broader teams.
- #6: The 2010 release for Microsoft is a very big release with a lot of rich capabilities. Quality is a key pillars for the release planning efforts and we aspired to achieve the above goals with the test offerings in the 2010 release by taking a fresh look at the problems.We want to align QA with the lifecycle to ensure that testers can add more value to the application lifecycle and are not looked at “necessary evil”. Having test assets co-located with the dev assets & project assets ensures visibility across the entire team.Create tighter integration across development & test teams to ensure that we can eliminate waste across the hand shake (or lack there of) that happens across devs & tests when it comes to bugs, builds, recreation of bugs etc. Enable highly leveraged QA teams ensures that teams are most efficient and are driving up productivity by focusing on key tasks and eliminating a lot of the waste that spans from boring repetitive tasks.
- #8: This is the fully animated slide.
- #12: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #13: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #14: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #15: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #26: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #28: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #29: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #30: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #43: http://www.wintellect.com/CS/blogs/jrobbins/archive/2009/06/16/how-does-vs2010-historical-debugging-work.aspxhttp://blogs.msdn.com/habibh/default.aspx
- #45: This would normally go in the Testing section but want to demonstrate it here during the build process.
- #47: http://blogs.msdn.com/mathew_aniyan
- #48: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #49: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #50: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #65: http://blogs.msdn.com/gautamg/archive/2010/01/07/platform-support-for-coded-ui-test-and-fast-forward-feature-of-test-runner.aspx
- #66: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #71: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #72: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #73: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment
- #81: Test Case Management using Microsoft Test and Lab Manager, a centralized testing toolProvisioning of multi-tier test environments and integrating it with build workflowFiling rich & actionable bug with minimal effort on the provisioned environmentCreating Coded UI Test using Action Log from Manual Testing & adding it run as part of build workflow on provisioned test environment