SubversionGuide.docx

SVN & Visual Studio Basics, Naming and
Use Standards

Developer’s Guideline

1/6/2016 4:11 PM Subversion Guide 2016 Page: 1

Topic Page

SVN & Visual Studio Basics
Installation and Access
Local Machine Setup
Working a JIRA Ticket
Local Machine Has Current Code
Developer is Ready to Make Changes
Peer Review and Tagging in SVN to Prepare for Promotion
Development Work Completed
If Adding or Modifying SSIS Packages
If Adding or Modifying a SQL Agent job
Example of link or script to do the If Adding or Modifying a SQL Agent job
Peer Review
Tag the Ticket into Code Review Approved
Name and Use Standard
RELEASE_NOTES
SCRIPTS
SQL_AGENT_JOBS
SSIS_PROJECT
Promotion Process – Code Review


Providing documentation to assist developers using Subversion is an extremely important focus
for us all. As part of this focus, we are developing this document, the Subversion Guide. The
Subversion Guide includes the basic information needed to use the tool as well as naming and
use standards that should be used during the process. This is a living document and will change
as we learn more about our process and challenges.

Installation and Access

Before you can use Subversion you must install the software on your local machine and gain access as
a user of the software:

1. Install of software

a. You can find the install file here
a. corp.twcable.comwest_LegacyLOSORANFILE01GG_ShareTechOpsApplications

b. The file name is TortoiseSVN1.8.8.25755x64svn1.8.10.msi

2. Access to use (if you are new member to BI Tech OPS group)
a. Contact Daniel Bowlin or Shashi Kadambi to assist.

Local Machine Setup

1. Start out by creating a folder named SVN in the root of your C drive or somewhere else you feel is
appropriate.

2. Inside that SVN folder create a folder for each TechOps project
a. BI_TOPS_DIRECT_OUTPUT
b. BI_TOPS_LOADS
c. BI_TOPS_MARKETING
d. BI_TOPS_METRICS

e. BI_TOPS_PMT
f. BI_TOPS_REALTIME
g. BI_TOPS_SBDB
h. BI_TOPS_SELF_BILLING
i. BI_TOPS_UTILITIES
j. BI_TOPS_WFDB
k. BI_TOPS_WFTXDB
l. BI_TOPS_WOA
m. BI_TOPS_WOBCDB
n. BI_TOPS_WODB
o. BI_TOPS_WORK


Working a JIRA Ticket

3. When you have a new ticket for work in one of these projects go inside the project folder and
create a folder with the ticket name (i.e. BI_TOPS_UTILITIES/IBTO1144)

Important note: The majority of you would work in the BI_TOPS_LOADS or
BI_TOPS_WODB projects.

4. Next right click the folder and choose TortoiseSVN > RepoBrowser


a. Put in the path to the project you are working on, to the branches folder.

Important note: SVN is case sensitive; please make sure all naming is done in the proper
CASE.   Be aware that the project can be different every time.


b. Click OK.

5. In the Repobrowser, you need to create a branch for your ticket.
a. Right click the trunk folder and choose Copy To


b. Enter the path to the branch and the ticket number you are creating.

Important note: The “branches” should be lower case, and the ticket number should be upper
case and include the dash.

c. Click OK and put in a message that includes the ticket number and anything else you want.


d. Click OK and the new branch for your ticket will be created. You may need to refresh the
branches a few times to see it, or close and reopen the Repobrowser.

A suggested way to refresh: Collapse branches, right click, then refresh, then expand
branch to see new folder

e. Once you can see the new branch right click on it and choose Checkout to create a copy
on your computer.


Important note: On the checkout dialog, make sure the Checkout directory is the one you
created on your computer.


f. Click OK and after the checkout is finished, click OK again.

g. Click Ok.

Local Machine Has Current Code

Now your local folder has a copy of the current code.

If you are modifying SSIS packages make sure you open the project from your
SVN/SSIS_Project folder/Ticket_Name.

Each ticket should have its own code repository on both the server and your local machine.

It is important to work each ticket code separately when you open it in SSDT.


Developer is Ready to Make Changes

Developer now opens Visual Studio to make changes.
Open the project:

Browse to the SVN/SSIS_Project/Ticket_Name/SSIS_PROJECT folder (i.e.
SVN/BI_TOPS_loads/ibto1212/SSIS_PROJECT/TOPSLoads.sln)


6. Once you have finished your work for the day, or when the entire change is complete, save and
build the project as you normally would.

7. Then go back to the file system and you will notice that the folder has a red exclamation indicating
that something has changed. Right click on the folder with the ticket number and choose SVN
Commit… to push the changes back up to the branch in SVN.


It’s a good policy to update SVN at least every day you make changes. This way if you are out
for an extended period, someone else can pick up the most current version of your changes
and begin work from there.

a. Put in a message.

Hint, click Recent Messages and use the same base message every time.


b. Click the ALL link (All is the default) and then click OK and your code changes should be
pushed back up to SVN.  Click OK to close the Commit dialog.

Peer Review and Tagging in SVN to Prepare for Promotion
Development Work Completed
When all development work is completed, and you believe you are ready for peer review, make sure
you COMMIT your work to SVN.
If Adding or Modifying SSIS Packages
If you are adding or modifying SSIS packages, please make sure to note in the Jira ticket or in the
release notes folder in SVN, each package that has been added or modified.

It is suggested: Developers put release or change notes right in SVN, then there is the history of
changes right in the source control.  This will be important during the promotion process.

If Adding or Modifying a SQL Agent job
If you going to add or modify a SQL Agent job the best approach is to supply a script for the entire
new version of the job with promotion instructions to rename or delete the old job and then execute
the script to create the new version of the job. However, it is not always practical to provide a full
script because of parallel changes to the SQL Agent job. If there is a dependency; setup a
dependency check within the job. If you do not provide a script, you will be expected to provide
explicit instructions including:
1.       The name of the new step
2.       The type of step it is (SQL, SSIS, etc.)
a.    The connection and path to the package if it is SSIS. Making sure to use the
production
       servers.
b.      The exact script if it is SQL.

3.   The environment or database connection to be used, making sure to use the appropriate
production environment because the environments and environment ID numbers are not
the
  same between DEV and PROD.
4.       Any environment overrides, or package specific parameters or connections.
5.       Where exactly in the job where the step should be placed
a.       For example: This step should be between the step labeled “start” and the step
labeled “Check HouseDim Load”
Then, using the repobrowser open the SVN branch that contains your work and tag it to
development.
Example of link or script to do the If Adding or Modifying a SQL Agent job


In the address window that opens change “branches” to tags/development and then add a suffix to
the ticket number of “_DEV_COMPLETED_01”.
Note: If you have tagged this work to development before and had to go back and do more work the
last 2 digits of the suffix will change appropriately to reflect the version i.e. 02, 03, 04, etc.

REMINDER: SVN is case sensitive, please observe the case used below in all instances.

Peer Review
At this point you are ready for peer review. Schedule that to be done.
Tag the Ticket into Code Review Approved
When peer review is complete send a note to the promotions team (currently Dan for BC & Shashi for
Resi) and ask them to tag the ticket into code review approved. We will then tag the ticket something
like this:

After the work is tagged to code review, you can fill out your promotion request document with
hyperlinks to the appropriate SVN objects and attach it to the ticket.
You can then refer to Section II. Ticket Information, Section III. Pending Production Promotions and
Section V. Links of the Promotion Process document to start and complete the promotion process per
your role on the project.


There are 2 types of SVN projects. An SSIS project, and a database project. An SSIS project in SVN will
have 4 subfolders RELEASE_NOTES, SCRIPTS, SQL_AGENT JOBS, and SSIS_PROJECT. A database
project in SVN will have 2 subfolders RELEASE_NOTES, SCRIPTS. The type of objects inside each
folder and the naming should following these conventions.
RELEASE_NOTES

There can be 2 documents in here. Both are optional, but when used should conform to the following
naming:
1. Development notes document. This may be used this to track the all the changes made to a
project and scripts. This is a simple checklist you can use to remember all the objects you
created or altered during the development process for the ticket.

Operations support opens and closes several projects every day. This is a great way to keep
track of things performed over time along with the notes you may put in the ticket itself.

i. Naming convention TicketName_ReleaseNotes
ii. Example IBTO604_ReleaseNotes
iii. Sample contents of file
1. –added new project parameter parm_DaysAgo
2. –changed package ImportEmplData.dtsx
3. –created alter script stagingHouseDim table

2. Deployment, or promote to production request. This is basically a copy of the promotion
request put in the ticket. Please follow this request for both the document you put in the
ticket itself, and this folder in SVN.
SCRIPTS


All scripts should be kept in database projects in SVN. Please do not use the scripts folder in the SSIS
SVN projects.Then you only have to worry about the new scripts you create.
1. Script naming conventions: ScriptAction_Database_objectname
a. There are a number of ScriptActions you can use
i. CreateSp  (create stored procedure)
ii. AlterSp  (Alter stored procedure)
iii. DropSp  (Drop stored procedure)
iv. CreateVw   (create view)
v. AlterVw  (Alter view)
vi. DropVw  (Drop view)
vii. CreateTbl   (create table)
viii. AlterTbl  (Alter table)
ix. DropTbl  (Drop table)
x. CreateFn    (create function)
xi. AlterFn  (Alter function)
xii. DropFn  (Drop function)
b. Database is an abbreviation
i. SB (dwh_trgt_SelfBilling)
ii. WF (dwh_trgt_workforce)
iii. WFTX (dwh_trgt_workforce_TX)
iv. WO (dwh_trgt_workorders)
v. WOBC (dwh_trgt_workorders_BC)
vi. ICOM (ICOMS_STG)
c. Object name is the actual object name without the schema prefix
d. To alter a stored procedure in the dwh_trgt_workorders database named
spUpdateHouseDim. The script name saved in this folder in SVN would be
AlterSp_WO_ spUpdateHouseDim

SQL_AGENT_JOBS

This is where scripts to create new SQL Agent jobs, or replace existing SQL Agent jobs should be
stored. It is not usually practical to alter a SQL Agent job so I typically drop the old one and create a
new one. So, I would recommend that all scripts here be drop & create scripts and the naming
convention simply be the name of the SQL Agent job following the standards for job naming that we
already use for TOPS jobs.

One additional note here is that new SQL agent jobs should all have a start logging step, and a finish
logging step. This is a process in itself, but if you look at existing jobs, should be fairly
selfexplanatory. Sometimes there are 2 sets of these logging steps in a single job. One set to mark
the start and end of the job itself, and the other to mark the start and end of a specific part of the job
upon which other jobs are dependent.  You can ask Shashi or me for help if you need.
Also make sure that any jobs you create have the proper dependency checks set up, and that any
other jobs that will depend on this new job have the proper dependencies set up as well.

SSIS_PROJECT

SSDT will name all the root level objects (project, solution, and subfolders, etc. so there is no need to
worry about that. Unfortunately the individual package naming in TOPS has been fairly inconsistent,
so at this point it will be very difficult to enforce any standard. SSIS Standards to use moving forward
are as follows:
Solution and Project Conventions
● When creating a new package, change the following values in the advanced properties of the
package
o Generate a new package ID
o Change name to reflect the name you gave the package
o Update the Creator name to yours.
● Projects/solutions should be saved in the appropriate project in SVN
● All Solution, Project and Package naming should be in camel case with no spaces.  Task and
Transform names can use spaces.  Please make all names meaningful to the task they are
performing.
● Please use the following prefixes for your task and transforms
Task/Transform Prefix
For Loop Container FLC

Foreach Loop Container FELC
Sequence Container SEQC
ActiveX Script AXS
Analysis Services Execute DDL ASE
Analysis Services Processing ASP
Bulk Insert BLK
Data Flow DFT
Data Mining Query DMQ
Execute DTS 2000 Package EDPT
Execute Package EPT
Execute Process EPR
Execute SQL ESQL
File System FST
FTP FTP
Message Queue MSMQ
Script SCR
Send Mail SMT
Transfer Database TDB
Transfer Error Messages TEM
Transfer Jobs TJT
Transfer Logins TLT
Transfer Master Stored Procedures TSP
Transfer SQL Server Objects TSO
Web Service WST
WMI Data Reader WMID
WMI Event Watcher WMIE
XML XML
DataReader Source DRSRC
Excel Source EXSRC
Flat File Source FFSRC
OLE DB Source OLESRC
Raw File Source RFSRC

XML Source XMLSRC
Aggregate AGG
Audit AUD
Character Map CHM
Conditional Split CSPL
Copy Column CPYC
Data Conversion DCNV
Data Mining Query DMQ
Derived Column DER
Export Column EXPC
Fuzzy Grouping FZG
Fuzzy Lookup FZL
Import Column IMPC
Lookup LKP
Merge MRG
Merge Join MRGJ
Multicast MLT
OLE DB Command CMD
Percentage Sampling PSMP
Pivot PVT
Row Count CNT
Row Sampling RSMP
Script Component SCR
Slowly Changing Dimension SCD
Sort SRT
Term Extraction TEX
Term Lookup TEL
Union All UNA
Unpivot UPVT
Data Mining Model Training DMMTDST
DataReader Destination DRDST
Dimension Processing DPDST

Excel Destination EXDST
Flat File Destination FFDST
OLE DB Destination OLEDST
Partition Processing PPDST
Raw File Destination RFDST
Recordset Destination RSDST
SQL Server Destination SSDST
SQL Server Mobile Destination SSMDST

Packages should be named according to the following standard
● Package name with no spaces or underscores
Connection conventions
● Connections should be named for the database, flat file, etc. only, no hostname
● All connections should use parameters.
Email task conventions
● All components of email should be variable.
o From, To, subject, message, etc.
● All email variables should use configurations
● The package naming convention for the variable is
o First the solution or project name
o Followed by an underscore and the variable name
Documentation conventions
● Task and Transform naming conventions should use the prefix naming conventions above
● Annotation should be added for every task or group of tasks with a common purpose
Variable conventions
● It is a best practice to use variables for as many of the package settings and values as possible.
● Please use camel case and do not use spaces in variable names

Parameter conventions
● All connection parameters should be prefixed by conn_
o i.e conn_dwh_trgt_workorders
● All non connections parameters should be prefixed by parm_
o i.e. parm_DaysAgo
● Parameter names. There are two primary types of parameters in use. Connection
parameters and runtime value parameters. Each starts with a lower case prefix indicating the
type of parameter and then continues with a camel case name. Since SSIS parameters and
variables are case sensitive sticking to these case rules will be important to make sure we set
up SQL Agent job input parameters properly.
● Connection parameters: Please prefix the name with conn_. For example in the existing
TOPSLOAD project there is a connection parameter named conn_DwhTrgtWorkforce that is
the connection string to the dwh_trgt_workforce db. In this case the underscore (_) is used as
a kind of name space separator. With that in mind the conn is separated from the database
name by an underscore to indicate the change from the parameter type (conn) to the
parameter specific name. The underscores are removed from the db name to stay consistent
with the idea of the name space separator.


Promotion Process – Code Review

The promotion team will take your promotion document from the ticket, and any release notes
in the ticket or in SVN and use that to move the objects into production.
1.     The trunk code will be compared to the new code to confirm all changes are valid, and
will not overwrite any other changes accidentally.
2.      All scripts will be executed on the production server as requested in the promotion
document.
3.      All SQL agent jobs will be added or modified as requested in the ticket.
a.      Promoter must validate the following after adding or altering a job
● The new job or step is showing no errors
● New steps are set to retry at least 5 times at 5 minute intervals if
appropriate.
● Any new jobs or steps must have valid paths to the PRODUCTION server,
global environments, parameters as needed
4.      If there are new SSIS packages the promoter will
● Create a local copy of the code review approved version of the code
● Create a local copy of the trunk
● Open the trunk version of the project in SSDT
● Add the altered packages/objects from the code review approved version of
the code using the release notes in the ticket or SVN to the trunk version of
the code
● Verify all connections and parameters in the new objects are not showing
errors after being added to the trunk
● Commit the trunk back to SVN
5.    After all the deploy/execute tasks have been done the promoter will tag the coder
review approved code to production MIG_COMPLETED_ with at date suffix YYYYMMD


6. Once the code has been tagged to production, the promoter will go back to the pending
promotions spreadsheet and mark the promotion complete and add any notes as
necessary.

SubversionGuide.docx

Recommended

More Related Content

Similar to SubversionGuide.docx (20)

More from Deborah Obasogie (20)

SubversionGuide.docx