IIS Logs to SQL Database using Powershell and Log Parser

Our IIS logs are ridiculously large, every time I am asked to find out some stats it takes me ages copying the files to my machine and running some log parser SQL on it.

So I thought it’s time to insert them into a SQL Database. I adapted a script which I mostly found here and used the following to do the import. At first I was getting the following error which I couldn’t figure out how to solve SQL table column “csUsername” date type is not compatible with SELECT clause item “csUsername” (type INTEGER)


This was being caused because some of the log files had the username as +12345678 or 12345678+ therefore no longer being an INTEGER and not being able to be imported.

To fix this look at line 27 of the following Powershell script to force csUsername to be a VARCHAR rather than an INTEGER

$ErrorActionPreference = "Stop"

Import-Module Pscx -EA 0

function ImportLogFiles(
    [string] $httpLogPath)
    If ([string]::IsNullOrEmpty($httpLogPath) -eq $true)
        Throw "The log path must be specified."

    [string] $logParser = "${env:ProgramFiles(x86)}" `
        + "\Log Parser 2.2\LogParser.exe"

    [string] $query = `
        [string] $query = `
        "SELECT" `
            + " LogFilename" `
            + ", RowNumber" `
            + ", TO_TIMESTAMP(date, time) AS EntryTime" `
            + ", s-ip AS sIp" `
            + ", cs-method AS csMethod" `
            + ", cs-uri-stem AS csUriStem" `
            + ", cs-uri-query AS csUriQuery" `
            + ", s-port AS sPort" `
            + ", TO_STRING(cs-username) AS csUsername" `
            + ", c-ip AS cIp" `
            + ", cs(User-Agent) AS csUserAgent" `
            + ", cs(Referer) AS csReferer" `
            + ", sc-status AS scStatus" `
            + ", sc-substatus AS scSubstatus" `
            + ", sc-win32-status AS scWin32Status" `
            + ", time-taken AS timeTaken" `
        + " INTO IisLogs" `
        + " FROM $httpLogPath"

    [string] $connectionString = "Driver={SQL Server Native Client 11.0};" `
        + "Server=<Database Server>;Database=<Database>;Trusted_Connection=yes;"

    [string[]] $parameters = @()

    $parameters += $query
    $parameters += "-i:W3C"
    $parameters += "-e:-1"
    $parameters += "-o:SQL"
    $parameters += "-createTable:ON"
    $parameters += "-oConnString:$connectionString"

    Write-Debug "Parameters: $parameters"

    Write-Host "Importing log files to database..."
    & $logParser $parameters

function Main
    $httpLogPath = "<Path to your logs>\*.log"

    ImportLogFiles $httpLogPath

    Write-Host -Fore Green "Successfully imported log files."


Now I just need to write an SSIS package which will update the database with the IIS log files from 3 server and two sites (daily), so I’ll be back to update you on how I got along.

Okay Automate copying IIS logs to SQL Database using Powershell, Log Parser and Task Scheduler post is now written, continue reading to find out how

Writable web directory in IIS 7.5

Recently I followed the advice re this topic on Stack Overflow, this had me add the IIS AppPool\DefaultAppPool or AppPool for your website to the security permission of the folder you wanted to write to.

This didn’t work for me.

What did work was when I allowed the local server Users group the access to modify the folder.


Migrating Oracle VB.net application from Windows 2003 to 2012

Lessons learnt whilst migrating a VB.net application from a Windows 2003 server to a Load balanced Windows 2012 server stack. Unfortunately it wasn’t as easy as deploying the old code on a new version of IIS.

One of the first issue that I had to solve was that the Oracle System.Data.OracleClient is now deprecated, therefore I needed to use oracle managed data access client Oracle.DataAccess.Client in order to connect to an Oracle DB using IIS 8.5

My web.config now looks something like this:

No longer available

The next task was to figure out where the oracle database would randomly drop its connection, I did this by debugging the code to VS2017,  I then had to open oracle datareader at several points, although the connection remained open at that same point when using the old oracle client.

A potentially dangerous Request.QueryString value was detected from the client. The application in question was creating one parameter in a report URL which was being formed using the ID from radio buttons selected. The HTML which was formed for the page changed between deploying and running the code on 2003 to how it appeared when it was being ran on a 2012 server. On the new server I was seeing that the parameter was passing in the wrong part of the HTML. Therefore it put in **** as a URL parameter. If at any point IIS or your browser sees this type of behavior it throws the above error.

How to fix this?

Two ways the easy way and the hard.

The easy way, which is not recommended

insert ValidationRequest=false into the page header for each ASP page as well as like this in your web.config, this shuts off the security feature and will leave your site open to scripting attacks:

No longer available

The hard way, figure out why the code is inserting a piece of HTML in as a parameter within a URL. It was doing this in my code due to the way that the HTML removed spaces, my code needed to determine where it was to grab the id of the radio button so was trying to locate a set point by reading in the ID of the radio button, because that was after value=2 > and it was looking for ‘value=’ + ID + ” “. Which it couldn’t find. as that was now written as value=2>.

But this won’t mean anything to you, as your code won’t be throwing an error because of the same reasons as above! Or at least for your sake I hope not!!

SSL for BI Publisher Standalone

We have a old installation of Oracle Business Intelligence which somehow has not been kept up to date – This happens through all businesses throughout the planet, the rule: If it works don’t touch it, applies to it.

So this server isn’t still running as HTTP, we needed to get it secured ASAP – 9 years late is better than never.

So how do you go about it?

Create a Java Keystore

For this you need to use the Keytool utility which comes with Java JDK / JRE on your server. See notes about this later on as this is a very outdated version of the Keytool utility.

Command to create a Java Keystore:

keytool -genkey -keyalg RSA -alias mykey -keystore mykeystore.jks

Enter a keystore password and remember it, I recommend that you use  Password Safe to generate and remember the password.

Create and Update secure-web-site.xml file

Make a copy of your existing file default-web-site.xml and name it secure-web-site.xml edit the new file and add secure=”true” to the end of the line as well as change your port, 9704 is HTTP as 9706 is the standard BI HTTPS port:

http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://xmlns.oracle.com/oracleas/schema/web-site-10_0.xsd" port="9706" display-name="OC4J 10g (10.1.3) Default Web Site" schema-major-version="10" schema-minor-version="0" secure="true">

Now add the line:

Before the closing tag

Register secure-web-site.xml file in server.xml

Towards the bottom of your server.xml file add:

Remove the line referencing the ./default-web-site.xml you no longer need HTTP access once you have HTTPS enabled. You’ll need to do something which I didn’t to have both HTTP and HTTPS URL’s accessible on a reboot the server will automatically delete the second

Add a local firewall rule for the new 9706 port

Use ‘Windows Firewall’ and look for an existing Inbound rule for 9704, copy this rule and change the port number to 9706.

Test if HTTPS URL is working

Save the changes and restart ‘Oracle BI EE OC4J‘ server in ‘Services’ to reflect the changes, that or reboot the server.

Browse to : https://:9706/xmlpserver/ using the servers available web browser or your local machine, as there could be corporate firewalls in the way still, these will not know about you new port 9706 so will need to be enabled.

A quick test from a command prompt on your local machine to see is:

Telnet YourServer 9706

Secure your new address with a certificate

Go back to where you created your mykeystore.jks file and delete the mykey from within it:

keytool -delete -alias mykey -keystore mykeystore.jks

Now follow the following steps:

  • Request your certificate from your certification authority
  • Complete your certificate request in IIS or elsewhere
  • Export your your certificate as a .PFX file
  • Extract from the PFX your Key and Pem files
openssl pkcs12 -in yourCertificate.pfx -out yourCertificate.pem
  • Convert yourCertificate.pem to .pkcs12 file
openssl pkcs12 -export -in yourCertificate.pem -out yourCertificate.pkcs12
  • Now and very importantly move the yourCertificate.pkcs12 file away from your Oracle BI server as this will have a very old version of Java Keytool on it, which does not support the -importkeystore command, mine was running JRE 1.4.2.
  • Also move the empty mykeystore.jks file to the same directory as the yourCertificate.pkcs12
  • Check your mykeystore.jks to check it is empty
keytool -v -list -keystore mykeystore.jks
  • On the server with JRE / JDK 1.6 or above, I ran this using JRE 1.8, run the following command to import yourCertificate.pkcs12 to the mykeystore.jks
"C:\Program Files (x86)\Java\jre1.8.0_131\bin\keytool.exe" -v -importkeystore -srckeystore yourCertificate.pkcs12 -srcstoretype PKCS12 -destkeystore mykeystore.jks -deststoretype JKS
  • Now check your mykeystore.jks again and you should now have the imported certificate
  • Move this mykeystore.jks to the Oracle BI server

Restart ‘Oracle BI EE OC4J‘ server in ‘Services’ to reload the application with the new certificate or reboot the server.

You should now have a lovely secured server, if at first it doesn’t show check it on a browser that you haven’t used yet or clear your cache.

Migrating Classic ASP websites from Windows 2003 IIS 6 to 2012 R2 IIS 8

I have recently been working on getting some major applications off a Windows 2003 platform, as this platform no longer meets security requirements, as we all know Microsoft stopped supporting the servers in July 2015.

To do this process I started by using Windows Server 2003 Migration Planning Assistant and Atlassian’s JIRA to map out the key steps and outcomes. I then worked with our Service / Storage and Networks teams to get the new servers spin up in VMware and get them to open oh’so important firewalls so the new servers have the same access as the old one.

It was then a case of modelling the structure on my local installation of IIS (Internet Information Services) there were some parts that weren’t straight forward as it seems this I will explain here. I hope for your sake that your company has already migrated away from Classic ASP code.

  • Global.asa – This file now needs to be in each application, rather than just one on your top level website.
  • In your AppPools – Make sure that you’ve set ‘Enable 32-bit applications’ to True.
  • Oracle – You still need to be install the Oracle 32-bit client, if of course your website is connecting to oracle databases using Microsoft ODBC for Oracle.
  • WebDeploy – Use WebDeploy from Visual Studio to the server to deploy your code, much easier than using a FTP server also saves having to deploy to each individual server, you can write a PowerShell script which will deploy to all your servers in one go.

Install the ASP roles and features:

    1. On the taskbar, click Server Manager.
    2. In Server Manager, click the Manage menu, and then click Add Roles and Features.
    3. In the Add Roles and Features wizard, click Next. Select the installation type and click Next. Select the destination server and click Next.
    4. On the Server Roles page, expand Web Server (IIS), expand Web Server, and then expand Application Development.Server role
    5. On the Server Roles page, select ASP.
    6. If the Add features that are required by ASP? dialog box appears, click Add Features. (This page appears only if you have not already installed ISAPI Extensions on your server.)Windows features
    7. On the Server Roles page, ASP and ISAPI Extensions should be selected. Click Next.Server role
    8. On the Features page, click Next.
    9. On the Confirmation page, click Install.
    10. On the Results page, click Close.

How to show your debug messages on screen… I’ll come back to this in a few days, it late at night and I’m too tired to continue explaining.

The above should get you started. Good Luck!

Right, I’m back to finish off the post.

Within IIS click the server name in the left hand tree, double click ASP within the IIS panel, open up the ‘Debugging Properties‘ and change ‘Send Errors To Browser‘ to True. You can do this as described above for the whole site and all it’s web applications or a application basis. To do this, select the application name on the left before double clicking on ASP.


You might also need to switch on Detailed errors for a particular error page. Let say your page is reporting 500 not found and you know that the page is there, a index.html file in the root displays correctly (a good test of sanity). To turn these on, Double click on Error Pages within IIS panel on the site or the application as above, then select the error code on the left and click ‘Edit Feature Settings…‘ on the right.


This then shows the following dialog box, where you click ‘Detailed errors‘.


This process creates or appends a web.config file in the root of your site or application which contains the following:


Microsoft OLE DB Provider for ODBC Drivers error ‘80004005’

If you’re getting this in a Classic ASP application and you’ve done all the above steps, then your Global.asa file has the wrong connection, maybe your oracle client doesnt have it in its tnsnames.ora file

Microsoft OLE DB Provider for ODBC Drivers error '80004005'

[Microsoft][ODBC driver for Oracle][Oracle]ORA-12154: TNS:could not resolve the connect identifier specified

/blah/blah.asp, line 26

Or if your getting something like the following then you’ve missed the ‘Enable 32-bit Applications‘ to True in the AppPool settings

Microsoft OLE DB Provider for ODBC Drivers error '80004005'

[Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified

/blah/blah.asp, line 141


Mailto subject / body in Android email client broken?

The following section of mailto: code wasn’t working on the default Android email client and Gmail client.

<div class="panel-body">
Please contact the <a href="mailto:it.helpline@random.com?Subject=No%20Tiles%20within%20Application&amp;Body=Please%20could%20you%20check%20my%20account%20@(Model.LoggedInUser.Id)%0D%0A%0D%0AMany%20Thanks" target="_top">IT Helpline</a></div>

The reason is down to a simple issue ?Subject and &Body both started with capital letters, replace these to be lowercase ?subject and &body and the following code works across all clients on all platforms.

<div class="panel-body">
Please contact the <a href="mailto:it.helpline@random.com?subject=No%20Tiles%20within%20Application&amp;body=Please%20could%20you%20check%20my%20account%20@(Model.LoggedInUser.Id)%0D%0A%0D%0AMany%20Thanks" target="_top">IT Helpline</a></div>

I’ve tested on :

Android : Default mail, Gmail, Nine

iOS : Safari, Gmail

Windows : Outlook, Gmail

Achieve ‘A’ rating on SSLABS server test using IIS 8.5 Windows 2012 R2

Do you want to reach the heights of having an A graded server on SSLABS server tests? 

First thing you need to do on a fresh install of Windows server 2012 R2 and IIS 8.5 is disable SSL3 functionality, this can be achieved by following this guide. Come back here once you have done this. Or if you trust me here’s my exported regedit key, which you can just import to your server.

Below is the Cipher Key string that I put into the Local Group Policy Editor. To do so click the Windows icon and type in gpedit.msc and press enter to launch the Policy Editor.  Open on the left the folder titled ‘Administrative Templates / ‘Network’  / ‘SSL Configuration Settings’ & double click on ‘SSL Cipher Suite Order‘ to open. Now Enable the SSL Cipher Suites and copy and paste the below list in. Click OK and restart your server, now run the SSL server test again to see the result.


Cipher suite from above all on one line for copying into GPEDIT.MSC


The following 2 Cipher’s currently only work on Windows server 2016 & IIS 10


*Note: I’ve included ‘TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256‘ & ‘TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384‘ in a separate code block below the main cipher keys, although these do not currently work on Windows 2012 with IIS 8.5 – I’m hoping that a patch with come through from Microsoft soon which enables them, as these will take the ‘A’ to an ‘A+’ rating, as they do when using Windows Server 2016 IIS 10.

So if you are using IIS 10 on Windows Server 2016 then add these to the top and removed the bottom two as all keys will not fit within the cipher field.