You snooze you lose... I lost....

I was working on a post about the new columns sql_memory_model, sql_memory_model_desc columns in DMV sys.dm_os_sys_info. These columns allow DBAs to programmatically identify if Lock Pages in Memory (LPIM) privilege is in effect at the service startup time.

I was setting it up, had one instance running with pages locked in memory and the other one without. Took some screenshots, then I noticed the MSSQL Tiger Team published the post
Developers choice: Programmatically identify LPIM and IFI privileges in SQL Server

So really there was no need for my post anymore. But what do I do with my beautiful screenshots?

Well not someone who likes to waste stuff, I decided to write just a couple of sentences and put the screenshots in this post and then link to the post by the MSSQL Tiger Team

Here is the query I ran to see instant file initialization is enabled, I ran this against two instances, one a SQL Server 2016 Service Pack instance, the other a SQL Server vNext instance

SELECT instant_file_initialization_enabled,* 
FROM sys.dm_server_services

You can see the output in the pic below, the vNext instance does not have instant file initialization enabled, the 2016 SP1 instance does have instant file initialization enabled

To see if Lock Pages in Memory is in use, you can run the following query

SELECT sql_memory_model,sql_memory_model_desc
FROM sys.dm_os_sys_info

You can see the output in the pic below, the vNext instance does not use Lock Pages in Memory, the 2016 SP1 instance does use Lock Pages in Memory

The sys.dm_os_sys_info DMV also has a new column that SQL Server 2016 SP1 does not have.. process_physical_affinity


To get all the details about this stuff in the pictures go read the post
Developers choice: Programmatically identify LPIM and IFI privileges in SQL Server published by the MSSQL Tiger Team

What to get your friendly DBA this holiday season

You have a DBA that you want to get something for this holiday season, but you have a hard time figuring out what to get him?
Here are 3 choices that will make a good present

The first two have to do with the availability of SQL Server on Linux. This is in preview right now, but it will be here before you know it.
I can already see the job postings a year fron now...

Install SQL Server on Windows as well as Linux
Manage Linux and Windows servers

And of course...  5 years experience installing SQL Server vNext on Linux  :-)

You go to the interview, then someone asks what does touch File1 do?

You get my point, SQL Server DBAs will need to learn Linux and they will need to learn this fast.

If your DBA likes to read, as a first step, I would recommend the following two books

Unix and Linux: Visual QuickStart Guide (4th Edition)
How Linux Works: What Every Superuser Should Know 2nd Edition

The Visual QuickStart book is for the total beginner, the How Linux Works book will go into more detail. Either book is perfect if the person has never touched a *nix system before.
Both books are nicely formatted, clear and concise, once the person is done with one of these books, the person can get himself/herself one of those 1000 page tomes


The next item is a book by an author whose previous work I have read and enjoyed

High Performance SQL Server The Go Faster Book by Benjamin Nevarez

I noticed that Benjamin Nevarez's latest book is available now on the Apress site for $9.99 in epub and pdf format, this price is valid through November 29, 2016

I didn't read this book yet, so here is what the website has to say about this book


What You Will Learn
Understand SQL Server's database engine and how it processes queries
Configure instances in support of high-throughput applications
Provide consistent response times to varying user numbers and query volumes
Design databases for high-throughput applications with focus on performance
Record performance baselines and monitor SQL Server instances against them
Troubleshot and fix performance problems

Here is a breakdown of the chapters

How SQL Server Works
Pages 1-19

Analyzing Wait Statistics
Pages 21-43

The Query Store
Pages 45-69

SQL Server Configuration
Pages 71-84

tempdb Troubleshooting and Configuration
Pages 85-99

SQL Server In-Memory Technologies
Pages 101-129

Performance Troubleshooting
Pages 131-154

Indexing
Pages 155-180

SQL Server Storage
Pages 181-196

There you have it...one of these will make a nice gift for your friendly DBA....

BULK INSERT with a file containing linefeeds gotcha

I blogged about this before but someone ran into this issue again at work so I thought it made sense to revisit this.

I prefer to use BULK INSERT or bcp as much as possible, this is especially true if all I need to do is dump the file into a table. If there is a need to do complex stuff,  I will use SQL Server Integration Services or Informatica. These days files are generated by all kinds of systems, these can be Windows, *nix, Mac, Raspberry Pis, IoT systems and many other systems.

If you get an error importing one of these files, your first course of action is to open one of these files in something like Notepad++ or Editplus.  Notepad++ and Editplus have more functionality and are many times faster than notepad. The nice thing about either of these is that you can see control characters.

See how you can see the linefeeds here? You cannot do that in notepad

So let's say you get a file where the row terminator is a linefeed, how would you specify that as a row terminator in BULK INSERT?

You can try n which stands for newline

BULK INSERT SomeTable
 FROM 'D:\JunkdrawImportMe.txt'
 WITH (FIELDTERMINATOR = 't',
 FIRSTROW =2,
 ROWTERMINATOR = 'n')

Nope, that doesn't work, you get 0 rows inserted


You can try r which stands for carriage return

BULK INSERT SomeTable
 FROM 'D:\JunkdrawImportMe.txt'
 WITH (FIELDTERMINATOR = 't',
 FIRSTROW =2,
 ROWTERMINATOR = 'r')

Nope, that doesn't work either, you get 0 rows inserted


What about l for linefeed?

BULK INSERT SomeTable
 FROM 'D:\JunkdrawImportMe.txt'
 WITH (FIELDTERMINATOR = 't',
 FIRSTROW =2,
 ROWTERMINATOR = 'l')

You get an error

Msg 4864, Level 16, State 1, Line 1
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 2, column 1 (SomeDate).



What about if you try a CHAR(10) which is a linefeed

BULK INSERT SomeTable
 FROM 'D:\JunkdrawImportMe.txt'
 WITH (FIELDTERMINATOR = 't',
 FIRSTROW =2,
 ROWTERMINATOR = CHAR(10) )

You get this error

Msg 102, Level 15, State 1, Line 5
Incorrect syntax near 'CHAR'.



How about if you embed it by using Dynamic SQL

DECLARE @cmd varchar(1000)
SET @cmd = 'BULK INSERT SomeTable
FROM ''D:\JunkdrawImportMe.txt''
WITH ( FIELDTERMINATOR = ''t'',
 FIRSTROW =2,
 ROWTERMINATOR = '''+CHAR(10)+''')'
EXEC(@cmd)

The dynamic SQL solution works, that was my original answer to this.

What about if we use the hex code for line feed 0x0a?

BULK INSERT SomeTable
FROM 'D:\JunkdrawImportMe.txt'
WITH (FIELDTERMINATOR = 't',
FIRSTROW =2,
ROWTERMINATOR = '0x0a') --CHAR(10) or ‘l’ does not work 
-- 0x0a is the hex code for linefeed CHAR(10)

Bingo, that works as well.  I actually prefer this, who wants to muck around with Dynamic SQL if there is an easier way....

That's all, keep this in mind next time you get a file with a linefeed and you are struggling importing that file

Interesting SQL related links for the week of Nov 25, 2016

Here are some interesting articles I read and tweeted about this past week, I think you will like these as well. If you are bored this weekend, some of these might be good for you to read



Released: Public Preview for System Center Management Packs for SQL Server

We are getting ready to update the SQL Server Management Packs family. Please install and use this public preview and send us your feedback (sqlmpsfeedback@microsoft.com)! We appreciate the time and effort you spend on these previews which make the final product so much better.

Please download at:

Microsoft System Center Management Packs (Community Technical Preview 1) for SQL Server

Included in the download are:

• Microsoft System Center Management Pack for SQL Server 2008/2008 R2/2012/2014/2016 (6.7.10.0)
• Microsoft System Center Management Pack for SQL Server Replication 2008/2012/2014/2016 (6.7.10.0)
• Microsoft System Center Management Pack for SQL Server Analysis Services 2008/2012/2014/2016 (6.7.10.0)
• Microsoft System Center Management Pack for SQL Server Reporting Services 2008/2012/2014/2016 (6.7.10.0)
• Microsoft System Center Management Pack for SQL Server Dashboards (6.7.10.0)

SQL Server 2016 SP1: Known issues

The SQL Product team has identified the following known issues which the customers may encounter after installing SQL Server 2016 SP1. The following post will help the customers  plan and prepare their environment for the following known issues while installing SQL Server 2016 SP1.

The Future of Programming (Bob Martin) -- we, programmers, are going to have to grow up and define our profession

Index Scans: Not Nearly As Bad As You Might Think

Using our good old buddy the StackOverflow demo database, let’s look at the Users table. I want to get the first 10 users in the table ordered by Id, the primary clustered key, which is an identity field.


Ooo – clustered index scan – that’s bad, right?

Not so fast. Click on the plan to view it in PasteThePlan.com, and hover your mouse over the Clustered Index Scan:

Source Control in SQL Server Management Studio (SSMS)

In the latest generation of SQL Server Management Studio, we moved to the Visual Studio 2015 Isolated Shell. While this provides SSMS a modern IDE foundation for many functional areas, it also had some consequences. Specifically, the integration with source control systems in SSMS no longer works the way it did in SSMS 2014 and prior.  Previously, one could install the Visual Studio MSSCCI provider and then integrate with various source control systems. Visual Studio 2015 does not support MSSCCI so that is no longer an option to use in SSMS.

Of course, the good news is that Visual Studio 2015 includes TFS and Git source control integration. With the move to VS 2015 Isolated Shell, SSMS should be able to use these packages as well, right? The answer is…yes…but! The issue for SSMS is that the TFS source control integration package VS provides also includes the entire suite of TFS integration features. If we include this package by default, SSMS will have Team Explorer in its entirety which includes things such as work item tracking, builds, etc. This doesn’t fit in the overall experience SSMS is designed for, so we aren’t going to include this package as part of SSMS. The full TFS integrated experience is included as part of SQL Server Data Tools which is designed for a more developer-centric set of scenarios.

That said, if source code integration is an important aspect of how you use SSMS, you can enable the Visual Studio packages manually.

Why is My Query Faster the Second Time it Runs? (Dear SQL DBA Episode 23)

Dear SQL DBA,

Whenever I create a complex view, the first time I use it in a query, the query is slow. The same thing happens after I alter the view. What is the reason behind this?

This is a great question — because when you ask this question, you’re about to discover a lot of interesting, useful things about how SQL Server runs queries.

There are two ways that SQL Server can use memory to make a query faster the second time you run it. Let’s talk about how the magic happens.