First of all, VMWare had set up some network adapters and placed them in a public profile. Enable-PSRemoting doesn’t like that. It was an easy google (bing?) to fix and Enable-PSRemoting succeeded.

I then issued this:

invoke-command -scriptblock { get-process | select-object -first 10  } -computer  localhost

Imagine my surprise when the result was this:

Could not find file 'C:\Windows\System32\WindowsPowerShell\v1.0\Event.Format.ps1xml'.
    + CategoryInfo          : OpenError: (:) [], RemoteException
    + FullyQualifiedErrorId : PSSessionStateBroken

I searched the internet for this, but only found one hit that was close, and that was a bug report for nuget.

It seems like the powershell engine that is running the remote payload is looking for a formatting file that doesn’t exist. To work around this, I simply copied an existing Format.ps1xml file (I chose Registry.format.ps1xml because it was the smallest), removed the signature from it, and changed the name of the view (so it wouldn’t change any output).

It’s not a big bug, and it’s a beta so I’m not worried. Just thought I’d share my workaround.

-Mike

The talk starts off with discussing times when it might make sense to avoid using the pipeline. Once you see the material, it’s makes perfect sense. He then moves to using PowerShell jobs to perform tasks, discussing the pros and cons of that approach. Finally, he talks about using the Runspace class to run separate PowerShell instances. It uses the classes, but still manages to be very readable, very approachable PowerShell. He provides several examples in each section (including a function that executes a PowerShell scriptblock with a timeout, something I’ve never seen before).

All in all, this was easily the best webcast on PowerShell I’ve ever watched. Unlike most Powershell videos I’ve seen, it wasn’t targeting a beginner, but someone who already knows the basics of scripting and wants to learn more. The techniques he presents are, as I have said, very straightforward and explained very well. I can already think of several examples of code that I’m probably going to be writing in the near future based on this presentation.

#module adder.psm1
function add-numbers($x,$y){
   return $x+$y
}

With that module, we can do the standard module stuff:

PS> import-module adder
PS> add-numbers 1 2
3

Ok, that was way too basic. Here’s something a lot closer to the topic at hand:

PS> $adder=import-module adder -ascustomobject
PS> $adder | gm

   TypeName: System.Management.Automation.PSCustomObject

Name        MemberType   Definition
----        ----------   ----------
Equals      Method       bool Equals(System.Object obj)
GetHashCode Method       int GetHashCode()
GetType     Method       type GetType()
ToString    Method       string ToString()
add-numbers ScriptMethod System.Object add-numbers(); 

PS> $adder.add-numbers(1,2)
Unexpected token '-numbers' in expression or statement.
At line:1 char:11
PS>  $adder."add-numbers"( 1, 2)
3

There are a several interesting things to notice about this example. First of all, note that the add-numbers function has become a scriptmethod on the $adder object. As the help topic for import-module states, the members of the custom object are the (exported) members of the module. When we try to call the add-numbers method, we find that our decision to use the noun-verb naming convention has bitten us. To use the method, we need to enclose the offending method name in quotes (both single and double work fine). Note that since this is a method we need to use commas to separate the arguments to the function.

A second thing to note is since this is a method, not a function, we can’t skip arguments.

PS> $adder.add-numbers(,2)

Note that we could definitely do

add-numbers -y 2

if we had used a normal import-module. Granted that in this case there would be no need to.

What if we try to fix the quotation issue by including an alias (say, AddNumbers) to the module and exporting it?

function add-numbers($x,$y){
   return $x+$y
}
new-alias addNumbers add-numbers
export-modulemember -Function * -Variable * -alias *

Here’s what we find:

PS> $adder=import-module adder -ascustomobject -force
PS> $adder | gm

   TypeName: System.Management.Automation.PSCustomObject

Name        MemberType   Definition
----        ----------   ----------
Equals      Method       bool Equals(System.Object obj)
GetHashCode Method       int GetHashCode()
GetType     Method       type GetType()
ToString    Method       string ToString()
add-numbers ScriptMethod System.Object add-numbers(); 

PS> get-alias AddNumbers

Capability      Name                             ModuleName
----------      ----                             ----------
Script          addNumbers -> add-numbers        adder

Hey! Our alias is missing. Unfortunately, it got imported into the global scope (possibly hiding another function). Note that I used the -force switch to make sure that we re-import it if it was already loaded.

When I first read about the -asCustomObject switch, I could see myself using this to import modules that had conflicting function names, and using the custom objects to call the methods in question. However, consider a function with a large number of switches or parameters. With an “-ascustomobject” object, you would need to specify all of the switches or parameters. Again, what about a function which used parametersets? As it turns out, scriptmethods don’t seem to use parametersets. Here’s function to demonstrate:

function test-psets{
param([Parameter(ParameterSetName="Set1")]$x,
      [Parameter(ParameterSetName="Set2")]$y)
      switch ($PsCmdlet.ParameterSetName){
         "Set1" {write-host "we're using Set1"}
         "Set2"  {write-host "we're using Set2"}
         default {write-host "don't know what parameter set we're in"}
      }
      Write-host "we had better be using $($PsCmdlet.ParameterSetName)"
}

Calling that function on a custom object looks like this:

PS> $adder=import-module adder -ascustomobject -force
PS> $adder."test-psets"(1)   #should use pset 1, since we're only using the first parameter
don't know what parameter set we're in
we had better be using
PS> $adder."test-psets"(1,2)  #shouldn't be valid, since they're different parametersets
don't know what parameter set we're in
we had better be using
PS> #Sanity check to make sure the function works
PS> import-module adder
PS> test-psets -x 1
we're using Set1
we had better be using Set1

PS> test-psets -y 1
we're using Set2
we had better be using Set2

PS> test-psets -x 1 -y 2
test-psets : Parameter set cannot be resolved using the specified named parameters.
At line:1 char:1
+ test-psets -x 1 -y 2
+ ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [test-psets], ParameterBindingException
    + FullyQualifiedErrorId : AmbiguousParameterSet,test-psets

So it seems that functions that utilize parametersets are going to be a lot less useful with -AsCustomObject imports.

As I mentioned, there are several examples floating around concerning creating new objects (or classes, depending on your perspective) using modules and this option. Given the drawbacks I’ve noted in this article, I think I’m going to stay away from that particular use case.

What do you think? Did I miss something important? Please let me know what your opinion is.

-Mike

If I were writing a script that needed to get a “bunch of things” from somewhere (perhaps several different sources) and return all of them, I might be tempted to do something like this. Please forgive my PowerShell pseudocode:

function get-stuff{
param($parm1 )
    $results=@()
    foreach ($source in $sources){
        $results += ($source | where { $_ -and "Some condition exists"  })
    }
    foreach ($source in $someothersources){
        $results += ($source | where { $_ -and "Some condition exists"  })
    }
 return $results
}

I’ve used several permutations of that kind of code using arrays of some sort to collect the results as I go along and eventually return the collection from the function. I’m not sure that there’s anything wrong with doing it this way. That is, I’m not sure that you’re likely to have issues with doing it this way.

On the other hand, it’s more idiomatic (i.e more in the style of the PowerShell language) to do something like this (again, pardon the pseudocode):

function get-stuff{
param($parm1 )
    foreach ($source in $sources){
        $source | where { $_ -and "Some condition exists"  }
    }
    foreach ($source in $someothersources){
        $source | where { $_ -and "Some condition exists"  }
    }

}

All I’m doing here is sending the output of the inner statements (which are pipelines) to the output stream of the function. Note that there isn’t any need for anything to accumulate the results into.  Using the output stream makes this function work more like the built-in cmdlets in PowerShell as it won’t be blocking the pipeline.

The only thing that I have against this code is that it goes against rule #2 that I wrote last time about writing values to the output stream. I said there that if you were going to write to the output stream, you should explicitly use write-output. We could modify the code above to use write-output, but that would involve using parentheses (around the pipelines), messing up the flow of the code, and even blocking the pipeline while the expressions in the parentheses were collected (as an argument to write-output).

That brings me to what I was saying about “another pointless function”. About a year ago I wrote a post about the identity function, which doesn’t really do anything except return the input. It is a really useful function for creating lists and such, allowing you to skip on providing a bunch of punctuation. It’s not a pointless function, but it’s not one that is getting much press, either. I was thinking about how to make the “pipeline” version of the code work nicely and not make it ugly and thought of the following function.

function out-output{
    process{ $_ }
}

Like the identity function (or ql, as I’ve seen it referred to), out-output doesn’t do anything but emit values that are provided. Out-output, however, gets its values from the pipeline rather than the argument list. This function allows us to be explicit about our intent to use the output stream.

function get-stuff{
param($parm1 )
    foreach ($source in $sources){
        $source | where { $_ -and "Some condition exists"  } | out-output
    }
    foreach ($source in $someothersources){
        $source | where { $_ -and "Some condition exists"  } | out-output
    }
}

I’m not sure if this is a good idea, and I know that it’s just adding a tiny bit of processing to the script. My thought is that making the operation of the script explicit is worth it in the long run.

What do you think?

-mike

Let me say that I’m very pleased with nosupportlinuxhosting and especially how they handled this outage.  I was informed promptly about the issue, the solution they were pursuing, and was notified when they server was back online.  For a “no support” company, I can definitely say that I’ve had better responses from them than the previous company that was handling the site.

I hope to have (most of) the posts back online in the next day or 2.  Fortunately I did have a backup from january (ouch), but that was just posts, pages, and comments.  If you receive some posts in your feed reader over the next few days, that’s probably me adding posts from the last 9 months or so.  I doubt I’ll be able to reconstruct the comments, so I apologize for that lack.

The easy “take away” from this is:  Make sure you have offsite backups for all of your systems.

The not so easy “take away” from this is: No, really, make sure you have offsite backups for all of your systems.

Maybe I’ll learn my lesson this time.

Mike

In most languages, if you see “return 1″ as the only return in a function, you can know that the function is going to the value 1 to the caller. In fact, I’m not sure I’ve ever seen a language that didn’t work that way. That is, until I found PowerShell. Generally speaking, the return statement works just as expected. In the absence of any statements writing to the output stream (with write-output) or “dropping” their values, “return 1″ will cause the caller to receive the value “1″. Using write-output is pretty obvious, and I’d recommend using it explicitly if you intend to add objects to the output stream (thereby including them in the eventual function value). Expressions that don’t capture their return values, however, are not quite so easy to spot.

For example, examine this code to add a parameter to an ADO.NET command object looks fine:

$cmd.Parameters.AddWithValue('@demographics','$demoXML)

This is a straightforward translation of one of the lines of code in the example code here.  The problem with the code is that AddWithValue not only adds a parameter, it also returns the parameter.  Since we didn’t assign it to a variable, cast it to [void] or pipe it to out-null, the output of this function (AddWithValue) gets added to the output of the function it’s in.

Several “add” functions in the .net framework follow this pattern, either returning the object that was added or the index of that object in the collection.  The DBConnection.Open method (inherited by SQLConnection, among others) returns the opened connection.  I’m sure that with time I could find more examples than I’d feel like sharing.

Another way that I’ve seen the output stream getting messed up is when a function uses strings to output information without using write-host.  For example, this function outputs “progress” information as it goes:

function get-filelength{
param($filename)
	"reading file"
	$len=0
	$file=get-content $filename
	foreach($line in $file){
	   "adding a line"
	   $len+=$line.length
	}
	return $len
}

I realize that this is not the best way to find the length of a file, but it works for the purpose of illustration. If you simply call the function (without assigning it to a variable), you’ll see a bunch of text, followed by the “value” of the function. The “returned value” of the function is clearly intended to be $len, but instead all of the text is included. The type of the function output is Object[], rather than a number.

There are some good reasons to use the output stream in a function (the next post will examine one such use), but it’s very easy to accidentally put something in the stream without intending to. When this happens, it often leads to unexpected results and long debugging sessions.

Here are my recommendations for avoiding this kind of error:

1. Use one of the “write” functions to provide text messages to the user (write-host, write-debug, write-verbose)
2. Use write-output if your intent is to add something to the output stream
3. Check msdn to see if the methods you use have output. If you’re looking at the C# version, methods that return void do not need to be “handled”. In the VB.NET version, they’ll be declared as Sub, rather than Function.
4. Be consistent with how you “handle” methods or functions whose values you want to discard. Options are to cast the result as [void] or pipe it to out-null
5. If you’re using write-output to put objects in the output stream, use return with no value (i.e., don’t return $val). Seeing a value returned explicitly implies that that is the only value returned.
6. If all else fails, use write-host to output the type of variables when debugging (or look at their values in the watch window if you’re using an IDE with a debugger).

Without further ado, here we go (now with linky-goodness!):

Tools I Use (note…these are all free!):

• PowerShell ISE (I know it’s part of the install, but I use it a lot!)
• PowerGUIScriptEditor (I still haven’t gotten the hang of PowerGUI itself)
• PowershellCommunityExtensions 2.0 (PSCX)
• SQLPowerShellExtensions 2.3 (SQLPSX)
• PrimalFormsCommunityEdition (free registration required) Download under “Free Tools/32-bit”

Books:

• PowerShellInAction,2ndEdition by Bruce Payette (just released yesterday, 5/18!)
• ProfessionalWindowsPowerShellProgramming: Snapins, Cmdlets, HostsandProviders by Arul Kumaravel et. al. (PowerShell 1.0, but still has good info about building hosts)
• MasteringPowerShell by Dr. Tobias Weltner
• PowerShellinPractice, by Richard Siddaway

In addition to these, there are any number of blogs (I really need to update mylist) that I subscribe to in google reader. Also, there’s a very active community on StackOverflow that will usually be able to provide direction if you’re not sure what’s going on.

As I said the last time, there are lots of other tools and books, but these are the ones I keep coming back to.

Let me know if you think I missed something super-important.

Mike

A friend of mine found this blog yesterday and commented on “PowerShellToolsandBooksThatIUse“. I thought that it would be good to update the list since it’s almost 2 years old.  The changes are mostly incremental: version changes, a few new items, a few I don’t use much anymore.

Without further ado, here we go (now with linky-goodness!):

Tools I Use (note…these are all free!):

●      PowerShell ISE (I know it’s part of the install, but I use it a lot!)

●      PowerGUIScriptEditor (I still haven’t gotten the hang of PowerGUI itself)

●      PowershellCommunityExtensions 2.0 (PSCX)

●      SQLPowerShellExtensions 2.3 (SQLPSX)

●      PrimalFormsCommunityEdition (free registration required) Download under “Free Tools/32-bit”

Books:

●      PowerShellInAction,2ndEdition by Bruce Payette (just released yesterday, 5/18!)

●      ProfessionalWindowsPowerShellProgramming: Snapins, Cmdlets, HostsandProviders by Arul Kumaravel et. al. (PowerShell 1.0, but still has good info about building hosts)

●      MasteringPowerShell by Dr. Tobias Weltner

●      PowerShellinPractice, by Richard Siddaway

In addition to these, there are any number of blogs (I really need to update mylist) that I subscribe to in google reader. Also, there’s a very active community on StackOverflow that will usually be able to provide direction if you’re not sure what’s going on.

As I said the last time, there are lots of other tools and books, but these are the ones I keep coming back to.

Let me know if you think I missed something super-important.

Mike

function convertfrom-SqlAgentDuration{
param($sql_duration)
    return New-TimeSpan -seconds ($sql_duration % 100) -minutes (($sql_duration / 100) % 100) -hours ($sql_duration /10000)
}

function convertfrom-SQLAgentDateTime{
param($sql_startdate,$sql_starttime)
    return (get-date -year ($sql_startdate/10000) -month ($sql_startdate/100 % 100) -day ($sql_startdate %100) -hour ($sql_starttime/10000 % 100) -minute ($sql_starttime/100 % 100) -sec ($sql_starttime % 100))
}

function convertto-SQLAgentDuration{
param([System.TimeSpan]$duration)
    return $duration.seconds+100*$duration.minutes+10000*($duration.hours+24*$duration.days)
}

function convertto-SQLAgentDateTime{
param([datetime]$date)
    return (new-object PSObject -property @{date=[int](get-date $date -uformat "%Y%m%d");time=[int](get-date $date -uformat "%H%M%S")})
}

Obviously, these are not high-performance functions, but given the volume of data you would be getting out of the SQL Agent, that shouldn’t pose any kind of problem. Here are some example usages:

PS>get-date

Monday, May 09, 2011 9:01:58 PM
_________________________________________________________________________________
PS>convertto-sqlagentdatetime (get-date) | ft -auto

    date   time
    ----   ----
20110509 210648
_________________________________________________________________________________
PS>convertfrom-sqlagentdatetime 20110509 210648

Monday, May 09, 2011 9:06:48 PM
_________________________________________________________________________________
PS>convertfrom-SqlAgentDuration 12345

Days              : 0
Hours             : 1
Minutes           : 23
Seconds           : 45
Milliseconds      : 0
Ticks             : 50250000000
TotalDays         : 0.0581597222222222
TotalHours        : 1.39583333333333
TotalMinutes      : 83.75
TotalSeconds      : 5025
TotalMilliseconds : 5025000

I hope these functions are useful to you. Let me know what you think.

If you’re reading this, the transformation went well.  It only took 30-ish minutes to accomplish.

Let me know if you find something missing or not working.

Mike

For those of you that are with me in the PowerShell camp, I have something that I’d like to discuss.  You probably have hundreds (dozens?) of scripts scheduled on multiple servers, possibly in multiple domains or geographical locations to perform things like these:

• Gather information about servers
• Generate reports about application usage
• Copy information from one place to another
• Validate security setup
• Start and Stop processes
• Scan log files for error conditions
• Lots of other things (you get the point)

How do you know that the scripts that you have written carefully and scheduled are actually running successfully?  At first, this seems like a silly question.  When you deployed the script, surely you ran it once to make sure it worked.  What could have gone wrong?

Here are some examples that come to mind:

• A policy was pushed which set the execution policy to Restricted
• The credentials you scheduled the script with have been revoked
• A file share that the scripts depend on is unavailable
• Firewall rules change and now WMI queries aren’t working
• The Task Scheduler service is stopped

You can probably think of a lot more examples of things that would keep scripts from working, but you get the idea.  I’ve given some thought to how to do this, but haven’t come to any real conclusions.  Obviously, having your scripts log results is helpful, but only if you monitor the logs for success/failure.  Also, if you have a script which is supposed to run every 10 minutes, it doesn’t help if you don’t get alerted when it only runs once in a day, even if  it runs successfully.  Also, if there is more than one person writing scripts, how do you make sure that everyone is using the same techniques to log progress?

Here are some of my thoughts:

• Use a “launcher” to run scripts (see below)
• Keep a database of processes with an expected # of runs per day
• Monitor matching start/end of scripts
• Log all output streams (example)

The first item in the list (the launcher) has been something I’ve been considering because it’s not trivial to run a PowerShell script in a scheduled task.  Even with the -file parameter which was added in PowerShell 2.0, it can involve a fairly long command-line.  With the added difficulty of trying to capture output streams (most of which are not exposed to the command shell) it becomes a process that is almost hard to get right every time.  Some features I’m planning for the launcher are:

• Load appropriate profiles
• Log all output streams (with timestamps) to file or database
• Log application start/end times
• Email owner of script if there are unhandled exceptions

I know this topic is not specific to PowerShell, but as Windows administrators get more used to scripting solutions to their automation problems with PowerShell (which I am confident that they are doing), it’s something that every organization will need to consider.  I’ll try to follow up with some posts that have some actual code to address some of these points.

Mike

P.S. I’m specifically not discussing “enterprise job scheduling” solutions like JAMS because of the high cost involved. I’d like to see the community come up with something a little more budget-friendly.