Thursday, July 8, 2021

Disabling SharePoint Online Site Creation for Users with Powershell

A friendly newer feature of SharePoint Online is that users can create their own SharePoint site without admin interaction. Note that this is different from restricting Office 365 group creation, this option lives under their SharePoint landing page. To disable this functionality in the web you can just do it via the SharePoint admin center (Settings > Site Creation), but the PowerShell method is harder to find.


The cmdlet you are looking for is: Set-SPOTenant -DisplayStartASiteOption $false


You must of course already be connected to SPO, whcih is out of scope of this post, see Here

Thursday, January 7, 2021

Hardening the Fortinet Single Sign On Agent Account

The Fortinet Single Sign On Agent (FSSO Agent) allows your Fortigate to map IPs to users which is useful/needed for web and other filtering policies. The agent gets installed somewhere (usually a domain controller) and then polls for logon events and keeps a database updated. This works fine, except that it wants to use a domain admin account for everything. If you are working in a hardened environment and want to restrict service account access then there are a few hoops to jump through, so here they are. This guide is not intended for getting your FSSO Agent integrated with your Fortigate, it is just talking about how to get it working with a hardened non-admin account.

Pre-Setup

In my environment we have group policies that make it so domain admins cannot login to workstations. We also deny interactive logon to service accounts. Ideally we don't want the FSSO account to be a domain admin anyways, so we won't be doing that. However, for the initial setup, this is needed. So to start i created a user called FSSO and added it to the following groups: Domain Users, Domain Admins, Service Accounts, and Event Log Readers. Note that Service Accounts is not a built-in group, that is one i created. You can skip this if your environment doesn't require it.

Installation

  1. Download the FSSO Agent from support.fortinet.com, this install was done with FSSO_Setup_5.0.0295_x64.exe
  2. Install the FSSO Agent on a domain controller. When prompted for Simple vs Advanced for your directory naming/structure, i use Advanced, but that is up to you and is out of scope of this post.
  3. Do not use the DC agent, use the option for Polling Mode with sub-option for Check Windows Security Event Logs

Adjustments

  1. Remove the FSSO account from the Domain Admins group
  2. Grant the FSSO account full control over the directory C:\Program Files (x86)\Fortinet\FSAE
  3. Grant the FSSO account full control over registry keys: HKLM\SOFTWARE\WOW6432Node\Fortinet\FSAE and HKLM\SOFTWARE\Fortinet\FSAE

Thursday, September 3, 2020

Office 365 Hybrid with Exchange 2010 Setup

Couldn't find a guide that actually gives you ALL of the steps required to setup a Hybrid organization with Exchange 2010 and Office 365, so here it is. This is not super technical, just the overall steps and things not mentioned/explained in other guides. This assumes you already have created a 365 tenant and added your domains to it.


Configure Azure AD Sync

Prep work: make sure you have added your primary email domain as an alternate UPN suffix (see Active Directory Domains and Trusts) and set all of your users UPNs to match their primary email addresses. This is not required but makes it cleaner


Next you need to synchronize your Active Directory to Office 365. Install the Azure AD Sync tool, and check the boxes for Password hash synchronization and Exchange hybrid deployment. You can check other boxes if you know what you're doing, otherwise best not to.


Important note: If you want to set this up with a group or OU based filtering, that's fine for testing. But be aware, any user NOT synced into 365 will be unroutable for sending email from 365 back to on-prem. So you really need to sync everyone.



Run the Hybrid Wizard

Grab it from https://aka.ms/HybridWizard or by logging into the 365 admin console, opening Exchange, and going to Hybrid. You should run this from an Exchange 2010 server.


When going through the settings, the defaults are typically going to be correct. If you need to adjust anything, go for it. The wizard will automatically create connectors for you and set up various things



Do the extra tasks they don't tell you about

  1. In the 365 Exchange Admin Console (EAC), navigate to Mail Flow > Accepted Domains and set your email domains to Internal relay. This is what allows 365 to route mail back to your on-premise environment
  2. If you haven't already, set your Azure AD Sync to push all of your Exchange users into Azure AD. You can verify this by going to Recipients > Contacts in the EAC, and you should see them all in there as type Mail user. If they are not, then stop and fix that. Again, this is for any user in Exchange, not just the ones you want to migrate.
  3. If you have assigned any licenses to users which grant Exchange access, you need to disable that or remove it. Users cannot be licensed for Exchange and also be migrated.


Migrate

At this point you can go to Recipients > migration and start a batch. Use the Remote move migration option and select whoever you want to migrate. I suggest just letting them auto-complete, but you can manually complete them if you wish. This takes a while. Once it's done, they may get a message in Outlook asking them to restart, however as long as they're on a newer client it should auto-reconfigure for 365 and you're done.

Tuesday, August 11, 2020

Setting up iOS14 mail with Azure AD App Consent

If you locked down your Azure AD to make it so users can't consent for approval on apps, then you may end up with iPhone/iPad users trying to setup their devices with iOS 14. Well, they can't consent, and the app isn't listed in the gallery or anywhere else. The user gets a message Need admin approval for Apple Internet Accounts. Instead of turning user approval back on, or trying to get a hold of an iOS 14 device yourself, you can still do the approval.

Head over to your Azure AD tenant and grab your Tenant ID (from the Overview tab). Then modify the following URL to add your tenant ID in:

https://login.microsoftonline.com/TenantID/oauth2/authorize?client_id=f8d98a96-0999-43f5-8af3-69971c7bb423&response_type=code&prompt=admin_consent

Heading to that URL should prompt for consent, and then your users can get in. Note that i omitted the redirect_uri parameter which would send you to the next site, so it may seem to hang after you approve. Just give it 10 seconds and close it out and have the user try.

Tuesday, October 22, 2019

Domain Controller Demotion Error: The operations master roles held by this directory server could not transfer to the following remote directory server.

In demoting a Domain Controller, ran into the following error:
The operations master roles held by this directory server could not transfer to the following remote directory server. 

FSMO Role: CN=Infrastructure,DC=ForestDnsZones,DC=domain,DC=local

Apparently during the last demotion, the roles for the DNS Zones weren't transferred (both for Forest and Domain). There is an easy fix, but it wasn't very straight forward to google.
  1. On the new Domain Controller (the one you're not demoting), open notepad and paste this into a file named fixfsmo.vbs:

    const ADS_NAME_INITTYPE_GC = 3
    const ADS_NAME_TYPE_1779 = 1
    const ADS_NAME_TYPE_CANONICAL = 2
    
    set inArgs = WScript.Arguments
    
    if (inArgs.Count = 1) then
        ' Assume the command line argument is the NDNC (in DN form) to use.
        NdncDN = inArgs(0)
    Else
        Wscript.StdOut.Write "usage: cscript fixfsmo.vbs NdncDN"
    End if
    
    if (NdncDN <> "") then
    
        ' Convert the DN form of the NDNC into DNS dotted form.
        Set objTranslator = CreateObject("NameTranslate")
        objTranslator.Init ADS_NAME_INITTYPE_GC, ""
        objTranslator.Set ADS_NAME_TYPE_1779, NdncDN
        strDomainDNS = objTranslator.Get(ADS_NAME_TYPE_CANONICAL)
        strDomainDNS = Left(strDomainDNS, len(strDomainDNS)-1)
        
        Wscript.Echo "DNS name: " & strDomainDNS
    
        ' Find a domain controller that hosts this NDNC and that is online.
        set objRootDSE = GetObject("LDAP://" & strDomainDNS & "/RootDSE")
        strDnsHostName = objRootDSE.Get("dnsHostName")
        strDsServiceName = objRootDSE.Get("dsServiceName")
        Wscript.Echo "Using DC " & strDnsHostName
    
        ' Get the current infrastructure fsmo.
        strInfraDN = "CN=Infrastructure," & NdncDN
        set objInfra = GetObject("LDAP://" & strInfraDN)
        Wscript.Echo "infra fsmo is " & objInfra.fsmoroleowner
    
        ' If the current fsmo holder is deleted, set the fsmo holder to this domain controller.
    
        if (InStr(objInfra.fsmoroleowner, "\0ADEL:") > 0) then
    
            ' Set the fsmo holder to this domain controller.
            objInfra.Put "fSMORoleOwner",  strDsServiceName
            objInfra.SetInfo
    
            ' Read the fsmo holder back.
            set objInfra = GetObject("LDAP://" & strInfraDN)
            Wscript.Echo "infra fsmo changed to:" & objInfra.fsmoroleowner
    
        End if
    
    End if
    
  2. Open an administrator cmd prompt at the location of the vbs
  3. Run it once each with ForestDnsZones and DomainDnsZones (or just specify the one you're getting in the error) like this:

    cscript fixfsmo.vbs DC=ForestDnsZones,DC=domain,DC=local
  4. You will need to fully close and re-run dcpromo to do the demotion, but it should go through now


Credit to this blog for the script.

Sunday, June 30, 2019

Setting up ConnectWise Manage with Duo Access Gateway via SAML

This will give you all the details on how to set up ConnectWise Manage with Duo Access Gateway via SAML. I could not find any documentation for the pairing of these two products, so there was a little bit of trial and error.


Pre-Requisites

  • A Duo Access Gateway (DAG) server already setup and ready to go.
  • A Duo account with Duo MFA or higher licensing. (the free version does not support this)
  • You must have an admin account to both CW:M and Duo (duh)

Recommendations/Caveats

  • The moment you save the SSO configuration in CW:M, it will remove all settings related to LDAP and existing 2FA with Google Authenticator. The CW documentation says it disables it, but no, it rips it out.
  • I recommend you sync your training database and do the setup there first. This way you can do it live, and also avoid issues caused by the point above
  • I am not going to train you on how to use the various products, this is just about the integration. I expect you already know how to create applications in Duo and add them to the access gateway.
  • This example is going to be for the training database in CW:M, so all URLs etc will reference that. The fake CW:M URL i will be using is manage.domain.com, and your DAG URL will be dag.company.com

Duo Setup

  1. Log into the Duo Admin Panel and create a new Application of type SAML - Service Provider
  2. Enter the following settings:
    • Service Provider Name: ConnectWise Manage - Training
    • Entity ID: https://manage.company.com/v4_6_release/auth/training/metadata
    • Assertion Consumer Service: https://manage.company.com/v4_6_release/auth/training/Acs
    • Service Provider Login URL: https://manage.company.com
    • NameID Format: urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress
    • NameID attribute: userPrincipalName (this can also be mail, just make sure it's an email address format that your DAG is pulling in from your identity source
    • All other settings: default
  3. Click Save Configuration
  4. Scroll down and set the name of the Application (i recommend setting it to the same thing: ConnectWise Manage - Training)
  5. Save and then click the link at the top to download the configuration file
  6. Head over to your DAG admin panel and upload the downloaded json file
  7. Download the certificate

CW:M Setup

  1. Navigate to System > Authentication (or Setup Tables > SSO Configuration)
  2. Set the following options:
    • Description: Duo Access Gateway (this can be whatever you want)
    • SSO Type: SAML
    • Login URL: copy and paste SSO URL from the DAG Admin
    • Identity Provider ID: copy and paste Entity ID from the DAG Admin
    • Identity Provider Certificate: upload the certificate you downloaded from the DAG Admin
    • Locations: check the box for the locations that this SSO applies to. In most cases it will just be All.
  3. Save and you're ready to test
  4. Use a different (incognito/private) browser window to test so you don't get locked out.
  5. Upon login it will redirect you to the SSO login. It will also handle user enrollments here so if this is a brand new org it's a good way to force your users to enroll
  6. Once you're done and tested, repeat the process for your production database. Replace training in the URLs with your company id. Create a new Duo Application for this, that way you will have it always working in both live and training.

Friday, February 8, 2019

Chrome, Remote Desktop Services, and Time Zone Redirection

If you're using RDS and have enabled the GPO for Time Zone Redirection, there is a small issue with Chrome that may cause some issues. For whatever reason, JavaScript date functions in Chrome will pull the system's time data instead of the user's. So if your server is in UTC and your user is in Eastern, Chrome's detection will come back as UTC. Firefox and Internet Explorer do not appear to have this flaw (i did not test with Edge).

To tell if you're affected, open the Chrome inspector (Ctrl+Shift+I) and go to the Console tab. If you just enter in new Date() you will see a preview of what the date is, including the time zone.

As far as i can tell, there is no workaround, if someone has one please let me know. I'll be setting up separate pools for users just for this.

Friday, November 16, 2018

Fortigate SSL VPN with LDAP User Authentication

The documentation was pretty inaccurate/unclear on a lot of this, so there was a bit of trial and error. This assumes you're already familiar with setting up SSL VPN and already have it working with local user accounts. Here's how you do it:


Pre-requisites

  • You need your SSL VPN portal and settings configured already
  • You should also have already created your SSL VPN policy (allowing from the SSL VPN interface to your LAN)
  • The above requires you to add a user or group already, you can re-use that group for the items below if desired

Set up LDAP Server

I'm using Active Directory, but you can use any LDAP based directory service. The example below assumes your AD domain is domain.local.

  1. Navigate to User & Device > LDAP Server
  2. Add a new server and enter the settings:
    • Name: this is the friendly name, i usually just put the hostname in
    • Server IP: obvious
    • Server Port: leave this default unless you know what you're doing
    • Common Name Identifier: This defaults to CN, which means Common Name. This is dumb. Your users are not going to log in as "Bob Smith", they're going to log in as bsmith or bsmith@domain.com. Set this to userprincipalname if you want them to use their UPN (username@domain.com) or maybe samaccountname if you want them to just use their username.
    • Distinguished Name: You can't use the Browse button until you fill out the rest of this page, but this can just be DC=domain,DC=local. Note: if you choose to scope this further than the domain as listed, your group (in the next section) must be in scope.
    • Bind Type: Regular
    • Username/Password: You can start off with a domain admin for testing, but ultimately you should create an unprivileged service account to use here
    • Secure Connection/Protocol: I used secure and selected LDAPS. If you don't have LDAPS then don't use it.
    • At this point you should be able to Test Connectivity and get a success.

Set up your group

  1. In Active Directory, create a group and add users to it. I called mine SSL VPN Users
  2. In the Fortigate, navigate to User & Device > User Groups
  3. Click on Create New
  4. Name the group the same as you created in AD (this isn't important, just a friendly name)
  5. Select Firewall as the type
  6. Under the Remote Groups section, click Add, select your LDAP server, and then search/select your group.
  7. Important: You have to right click on it and select Add Selected. After that, hit OK, not before.
  8. Hit OK again to save the group
  9. Configure your SSL VPN firewall policy to use this group for authentication

Testing

At this point you should be done, because you already set up your SSL VPN, right? You should be able to log in as the user now, you can go to Log & Report > VPN Events to see what the error is if you're not able to log in. You can also use the command diag test authserver ldap "YOUR LDAP SERVER NAME" youraccount@domain.com yourpassword to do a direct test.

Wednesday, October 31, 2018

Apple Open Directory - Options Greyed Out after IP Change

I recently had to move a few Apple servers from one site to another for a client, and after the move the Open Directory was half-working (opens, lets you view users, but can't add/remove/edit). There were a few no-brainer fixes that i had to do initially, but at the end of the day the final fix was pretty much a bandaid.

Overall obvious steps:

  1. Make sure your hosts file (/etc/hosts) is referencing the right IPs, in case anything changed.
  2. Make sure the system is set to the right IP(s) for DNS.
  3. Make sure the DNS Server records are updated to the new/correct IPs for both the forward and reverse zones.

The bandaid fix for me

In my case, when authenticating within the Workgroup Manager, it would lag and then eventually log in. Nothing in the logs about what was wrong, but the New User button was greyed out, as well as all of the fields on the actual accounts. For whatever reason i could reset passwords, not sure why. The ultimate fix was to add an additional network interface to the server, and set the IP address to the old IP of the server. Yep.

Troubleshooting Steps

  • First i fired up tcpdump and did a filter of "port 53" to see if any DNS requests were going out or coming back in incorrectly. Didn't find anything there.
  • Next i did tcpdump "host 10.x.x.x" (with the old IP of the server) on a hunch, to see if it was trying to send any traffic there. And yep, it was, on port 389 (LDAP).
  • Knowing that, i did a grep for the IP in /etc and a few other places but couldn't find it, assuming it's in a binary DB somewhere or something. So the only solution is to make the traffic actually connect.
  • In Networking, i added a new interface under the existing Ethernet, gave it the same IP of the server before the move, but with subnet mask 255.255.255.255. This allows the IP to respond without interfering with any other IPs on the subnet.
  • After making that change, the directory connected and started working instantly.

Thursday, August 30, 2018

Migrating to Office 365 from BAE Systems / Silversky Hosted Exchange

This was a fun one... not an easy process to figure out so hopefully this helps someone. To migrate away from the BAE Systems aka Silversky Hosted Exchange platform, you need to get a few things:
  1. Your Office 365 organization should already be set up and ready to go (obviously)
  2. An admin account in Hosted Exchange that has Full Access to the mailboxes you wish to migrate, and is a licensed user (has an Exchange package)
  3. Either an Outlook installation you can use, or the testconnectivity.microsoft.com website will also do.
  4. Access to the BAE Systems Exchange Provisioning Console (it's a java app)

Office 365 Setup

At this point you should have your organization created, domain added, and at least one licensed user (admin account is ok). That part of the setup is far out of scope for this guide.

Getting Admin Access

If you have just a few mailboxes, then you can just do this yourself. I highly recommend you test this with one or two mailboxes first, as i have had the built-in cusrpt@ account not work correctly.

For each user:

  1. Open the user account in the provisioning console
  2. Go to Packages, and edit the Exchange package
  3. Launch the Mailbox Rights Management window
  4. Add your admin account and grant it Full Access
  5. Save all the way out, rinse and repeat

If you have a ton of mailboxes, then you may need to call support. You can launch multiple instances of the provisioning console to pipeline the requests, but i just did 500 users, so that wasn't happening. In that case, you can call support and get them to run a Powershell command to grant you access. It may take a little work to get them to escalate it, and they also may want to charge you for it. I'm not going to go over the details of the command here, you can google that one.

Starting the Migration Batch

  1. Open the Exchange admin center, and head over to Recipients/Migration
  2. Create a new Migration batch, "Migrate to Exchange Online"
  3. Set the batch type to Cutover
  4. Enter the account credentials that you set up in the previous section (the one with access to all the mailboxes)
  5. It will probably give you an error at this point, that's why this guide exists. You now need to enter the Exchange server and RPC proxy server. Here we go...
    The easy part first: set the RPC proxy to exchange.postoffice.net. Now the hard part. Use either the testconnectivity.microsoft.com website, or (for the security conscious) your Outlook's Autodiscover feature. To use the former, just go to the website and enter the info, doing an Autodiscover test. To use the latter, ctrl+click on your Outlook icon, go to Test E-mail Autoconfiguration, enter the credentials and un-check the Guessmart options.

    At this point, regardless of the option, you will end up with the Autodiscover XML output. In here, you will see a section that says:

    <Protocol>
      <Type>EXCH</Type>
      <Server>2bd60169-a9ac-4f15-bdd4-dea1a11129a7@yourdomain.com</Server>
    

    You need to copy/paste the contents of the Server tag (the part that looks like an email address, including the @yourdomain.com). This is what you use for the Exchange server field.
  6. Once you've done that, you do need to use the advanced/more option and set Authentication to Basic and Mailbox Permission to Full Access and hit Next
  7. Name your batch, select the notification email (aren't you glad i told you to have at least one licensed user?), and you can start.
At this point, you're migrating. You can monitor the batch status from the console like you normally would. If doing a large number of mailboxes, use Powershell.

Monday, July 2, 2018

Fortigate - Policy Routing to VPN Tunnels

I have found that FortiOS v6+ has slightly different/broken functionality when handling policy routes across VPN tunnels (phase1/2-interface). In v5 you could create a policy route pointing to the tunnel interface and leave the gateway address set to 0.0.0.0, and everything worked fine. In v6 however, it appears leaving it as 0.0.0.0 is Fortinet code for please don't use me, i'm stupid.


The fix is pretty simple, assign an IP on both sides on the tunnel interface (just a /30 is fine, i.e. 10.255.255.1/30 and 10.255.255.2/30). Then in the policy routes, reference that IP as the gateway, and it should start working.


Sample Config:

(Note that in this case, the tunnel was to route all internet bound traffic across the VPN, so yours may look different)

Firewall 1

config system interface
    edit "Corp"
        set vdom "root"
        set ip 10.153.153.2 255.255.255.255
        set type tunnel
        set remote-ip 10.3.153.1 255.255.255.252
        set interface "wan1"
    next
end

config router policy
    edit 1
        set input-device "internal"
        set src "10.2.53.0/255.255.255.0"
        set dst "0.0.0.0/0.0.0.0"
        set gateway 10.153.153.1
        set output-device "Corp"
    next
end

Firewall 2

config system interface
    edit "Remote"
        set vdom "root"
        set ip 10.153.153.1 255.255.255.255
        set type tunnel
        set remote-ip 10.3.153.2 255.255.255.252
        set interface "wan1"
    next
end

config router policy
    edit 1
        set input-device "internal"
        set src "0.0.0.0/0.0.0.0"
        set dst "10.2.53.0/255.255.255.0"
        set gateway 10.153.153.2
        set output-device "Remote"
    next
end

Monday, May 7, 2018

Automatically Rotating Group Policy Logon Messages

This is a pretty common task for various types of compliance, where you use the logon message via GPO to display security notices or other company facts. But you can't just have a stale message, it needs to be rotated to keep it fresh. I got tired of doing mine by hand, so i wrote a script to do it. Unfortunately the PowerShell Group Policy module has pretty poor documentation, and i'm not even confident it's capable of doing this. That said, some group policies use files in SYSVOL, while others store their data in AD. In this case, it's in a .inf file, so here is the powershell script:

$basepath="\\yourdomain.local\sysvol\yourdomain.local\policies\{CF267D2E-F5BE-46D9-85B3-58125FEFB1CF}\machine\microsoft\windows nt\secedit"
$tmpfile="$basepath\tmp.inf"
$tplfile="$basepath\GptTmpl.inf"
$bakfile="$basepath\GptTmpl_bak.txt"
$notices=@(
    "Your first notice.",
    "Your second notice.",
    "Etc."
)

$phrase=$notices[(get-random -Maximum ([array]$notices).count)]
$phrase=$phrase -replace ",","`",`""  #the .inf wraps commas in quotes when it's part of the string

new-item -path $tmpfile -ItemType file -Force | out-null
foreach ( $line in get-content $tplfile ) {
    if ( $line -match "LegalNoticeText" ) {
        "MACHINE\Software\Microsoft\Windows\CurrentVersion\Policies\System\LegalNoticeText=7,$phrase" | out-file -filepath $tmpfile -append
    } else {
        $line | out-file -filepath $tmpfile -append
    }
}
move-item $tplfile $bakfile -Force
move-item $tmpfile $tplfile

Just get the GUID of the GPO you're using from GPMC and replace it in the $basepath, and you're set. Run that as an account with permission to that path and it will automatically rotate it.

Wednesday, April 25, 2018

Exchange 2010 Installation Fails with Error "Provisioning layer initialization failed"

For a customer with an older environment, needed to re-build a DAG member and ran into this issue. Upon installation of the management tools, even with a fresh SP3 download, the install would error out. The guts of the error messages were this:


[04/24/2018 22:21:56.0372] [2] [ERROR] Provisioning layer initialization failed: '"Scripting Agent initialization failed: "File is not found: 'C:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml'.""'
[04/24/2018 22:21:56.0372] [2] [ERROR] "Scripting Agent initialization failed: "File is not found: 'C:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml'.""
[04/24/2018 22:21:56.0372] [2] [ERROR] "File is not found: 'C:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml'."
[04/24/2018 22:21:56.0388] [2] [ERROR] Provisioning layer initialization failed: '"Scripting Agent initialization failed: "File is not found: 'C:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml'.""'
[04/24/2018 22:21:56.0388] [2] [ERROR] "Scripting Agent initialization failed: "File is not found: 'C:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml'.""
[04/24/2018 22:21:56.0388] [2] [ERROR] "File is not found: 'C:\Program Files\Microsoft\Exchange Server\V14\Bin\CmdletExtensionAgents\ScriptingAgentConfig.xml'."
Turns out, this is related to the PowerShell cmdlet Extension called Scripting Agent, which is enabled by default. Many posts online say you can just go to the path listed above and rename the .sample file to fix it, but that doesn't actually fix the management tools. In actuality, you need to disable the agent, run the install, then (if desired) re-enable the agent.


  1. Open a the EMS (Exchange Management Shell) on another exchange server in the org and run:
    Disable-CmdletExtensionAgent "Scripting Agent"
  2. Fully install Exchange
  3. Copy C:\Program Files\Microsoft\Exchange Server\V14\bin\CmdletExtensionAgents\ScriptingAgentConfig.xml from another exchange server to the same path on the new server
  4. Re-enable the agent:
    Enable-CmdletExtensionAgent "Scripting Agent"
I suppose not much Microsoft could do to work around this without releasing a new service pack, but still frustrating.

Friday, December 8, 2017

ConnectWise Manage Report Writer - Repeaters and Subtotals

If you're reading this you're probably aware that the support/documentation provided by ConnectWise for more difficult tasks in Report Writer is a bit... lacking. I ran into a few issues on Saturday and while i figured them all out the same day, i did a CW chat and submitted a ticket and have yet to hear back (4 full business days later). Basically, if you want to use Repeaters and/or Subtotals, there are very specific constraints that you need to keep in mind. This post is not for someone just getting into report writer, you need to already know how to build queries, use joins, and more.

Repeaters

These are a great solution to not having to use a sub-report. That said, they're not very intuitive and are in fact pretty basic in nature. The gist of it is that once you start the repeater, the first column you reference and all subsequent ones (unless you hit a nested a repeater) are repeated until the initial column changes. The documentation flip flops between using [repeater], <!--[repeater]-->, and <repeater>. The latter is never correct. The first two can be used interchangeably, but if you're wrapping this around HTML content then the middle option is the correct one.

As an example, say you are making a report that lists a client's agreements with their additions. The data in table form may look like this:

Agreement NameAnniversaryAddition NameAddition QtyAddition PriceExtended Price
Managed Services1/1/2019Antivirus30$5.00$150.00
Managed Services1/1/2019Agent30$1.00$30.00
Managed Services1/1/2019Spam Filtering35$3.00$105.00
Telecom Support2/1/2020Handset Fee20$10.00$200.00
Telecom Support2/1/2020PBX Maintenance1$100.00$100.00

As you can see, the agreement name is duplicated for each time you have an addition on it, because this is how SQL works. So in Report Writer, on the Fields tab, you would click Design Form and then (if you're sane) edit the HTML directly. The code to put this together would look something like this:


[Company_Name]
<!--[repeater]-->
  <h3>[Agr_Name] - [Agr_Anniversary]</h3>
  <table>
    <tr><th>Item</th><th>Quantity</th><th>Cost</th><th>Ext Cost</th></tr>
    <!--[repeater]-->
      <tr>
        <td>[Line_Desc]</td>
        <td>[Qty]</td>
        <td>[Cost]</td>
        <td>[Ext]</td>
      </tr>
    <!--[/repeater]-->
  </table>
 <hr />
<!--[/repeater]-->

Note that there are nested repeaters. The first level is for the Agreement Name and Amount, the second level is for the items associated with each one. The resulting output looks something like this:


Company Name


Managed Services - 1/1/2019

ItemQuantityCostExt Cost
Antivirus30$5.00$150.00
Agent30$1.00$30.00
Spam Filtering35$3.00$105.00

Telecom - 2/1/2020

ItemQuantityCostExt Cost
Handset Fee20$10.00$200.00
PBX Maintenance1$100.00$100.00


Subtotals

Now let's say that you want to add up all of the addition amounts on the agreement and show that total. Since you still want the itemized list, you need to use the Subtotal Decorator. The CW Manage documentation flip flops on whether this is @Subtotal or #Subtotal, don't ask me why. The correct usage is @Subtotal. There are however a few constraints to consider:

  • Subtotal only makes sense in a repeater. If your data looks how you want it and you're not using a repeater, then this should be done in the query, not in the report designer.
  • You must check the VG (visual group) box on every column not part of the repeater that will be subtotaled. This will force them to be sorted as well, but that's not a big deal.
  • You must check the Add Subtotals box at the bottom of the Fields tab.
  • CW will tell you that you can only use Subtotal on one field. This is incorrect, you can subtotal as many fields in a repeater group as you want.
  • CW neglects to tell you that the column you are subtotaling cannot have spaces OR underscores in it.

Using the repeater example above, here is how you would add a subtotal of the overall ext costs of the additions:


[Company_Name]
<!--[repeater]-->
  <h3>[Agr_Name]</h3>
  <table>
    <tr><th>Item</th><th>Quantity</th><th>Cost</th><th>Ext Cost</th></tr>
    <!--[repeater]-->
      <tr>
        <td>[Line_Desc]</td>
        <td>[Qty]</td>
        <td>[Cost]</td>
        <td>[Ext]</td>
      </tr>
    <!--[/repeater]-->
    <tr><td colspan="3">Total:</td><td>[Ext@Subtotal]</td></tr>
  </table>
  <hr />
<!--[/repeater]-->

It's nice to note, the Subtotal can go above or below the repeater group. If you want to subtotal more than one column, just add that column with the @Subtotal decorator wherever you need.

Wednesday, November 15, 2017

Migrating VMkernel adapters on vSphere Distributed Virtual Switches

A chicken/egg scenario can sometimes occur when setting up DVS (distributed virtual switch) in a vCenter cluster. The issue being your vCenter is connected to your host over its' vmk0, so any network changes have to be confirmed or else they will be rolled back. In my case, i wanted to change the links to have my management network be a tagged VLAN instead of native. The problem here is that i would need to make the change in vCenter, then change the tagging on the switch ports, then hope that it picked up the changes. In almost all cases, it didn't, and reverted the settings. In a few cases i got lucky with the timing and it worked, but most of the time it didn't.


The solution is to set the Advanced vCenter setting of config.vpxd.network.rollback to false, then make the change. This causes it to not roll back the changes, so you better be sure that your configuration is correct. This should of course be set back to true when you're done.

Friday, June 23, 2017

Labtech and Corrupted Scripts

Update: The latest patch fixes this, or at least handles it more gracefully. v110.387 (Patch 15) is what is needed, otherwise see below.

Labtech (or Connectwise Automate, which i refuse to say outloud), has a current bug where you can quite easily corrupt a script. Since the scripts are not traditional text files, and are stored encoded in the database, there is no easy way to fix it if something does get corrupted. I'll outline how this happens, and how you can fix it, since their support has indicated that this is a low priority issue to them.

The Issue

For various reasons a script can get corrupted. When i say corrupted, it's all still there, but something is preventing it from getting loaded. For example, the other day my coworker added a new EDF and then asked me to take a look at a script. When i loaded it, the script functions did not show the EDF name but just showed the ID. Though i advised against it, we ended up making some changes and saving the script. After that, the script would no longer load with the error "Error loading script:Syntax error: Missing operand after '=' operator." (see below):



Now, you'd think that Labtech would still load the script and force you to fix it... NOPE. Any work you've put into that script is now lost, you need to restore from backups. This means either restoring your entire database, or grabbing the nightly .sql backup files, editing to grab the right data, and replacing via SQL. But what if you just put hours of work into the script and backups haven't run yet? Well, you're SOL in this case. Or are you?

The Fixes


Restoring from backup

If you just want to go to last night's backup, then the fix is not too difficult. You will need adequate knowledge of running SQL queries against the Labtech server, and the text editor of your choice.

  1. Take a new backup, in case you mess something up.
  2. Grab the backup that you want to restore. These are usually located in C:\Program Files\LabTech\Backup. You can go into the Tablebase folder or extract one of the zip files if you need an older copy. The file you need is lt_scripts.sql. Make a copy of this in a spot where you can edit it.
  3. Determine the ID of the script that is corrupted. Easiest method here is to show IDs in the Labtech Client (Tools > Show ID's). The ScriptID is the number in parenthesis next to the script in the Navigation Tree. For the sake of this guide, we will be using ID 5986.
  4. Edit the .sql file, i used notepad++. Search for the ID of the script, you should see something that looks like this. The first red square is what you're looking for with the script ID, the second is the actual content that you need to use. You want everything between the apostrophes, in my example starting with H4sI and ending in AAA===. I had to blur a lot of this for obvious reasons, but you should get the gist of it.
  5. Open your preferred SQL editor (Labtech staff usually install SQLyog on the server, you can also use the CLI or MySQL workbench)
  6. Craft a query that reads:
    UPDATE lt_scripts SET ScriptData='H4sI......AAA==' WHERE ScriptID=5986;
    (replace 5986 with your ScriptID, and paste the FULL content of what you selected into the ScriptData portion of the statement.)
  7. Run the query, and it will replace the body of the script with your backup copy.
  8. At this point you should be able to edit the script, no Refresh/Reload needed
  9. If not, you may need to go to an earlier backup

Fixing the script in-place

This is definitely the more difficult route. If your backup from last night is good enough then i'd use the previous section to restore from backup. Otherwise, read on. This requires knowledge of running SQL queries, minor XML knowledge, as well as the ability to perform Base64 encodes/decodes and gzip compressions/decompressions. I did this on a linux server, however you can also do it with other tools. I will gloss over various tasks that are covered in the previous section for the sake of brevity.

  1. Take a new backup, in case you mess something up.
  2. Extract the ScriptData for this script using this query:
    SELECT ScriptData FROM lt_scripts WHERE ScriptId=5986;
  3. Base64 Decode the result. You can use an online tool, or on a Linux shell:
    # base64 -d > output.gz
    After running that command, paste the data in and press Ctrl+D. However you do it, you need to end up with a .gz file containing the base64 decoded content.
  4. Unzip the file. On a Linux shell you can do:
    # gzip -d output.gz
    or
    # gunzip output.gz
  5. output is now an XML file containing the script. Edit this with a text editor (i use Notepad++) and find the section with the error. If you note, when you open the script editor and it errors out, it does load at least a few lines before dying. The last line it loads is where you want to start looking. If this has a variable name or something you can search for, use that. In my case of a missing EDF, i ended up with a section that looked like this:
    <ScriptSteps>
        <Action>2</Action>
        <FunctionId>103</FunctionId>
        <Param1 />
        <Param2>%computerid%</Param2>
        <Param3>DuoDefaults</Param3>
        <Param4 />
        <Param5 />
        <Sort>8</Sort>
        <Continue>1</Continue>
        <OsLimit>0</OsLimit>
        <Indentation>1</Indentation>
      </ScriptSteps>
      <ScriptSteps>
        <Action>2</Action>
        <FunctionId>103</FunctionId>
        <Param1>611</Param1>
        <Param2>%computerid%</Param2>
        <Param3>APIHostnameComputer</Param3>
        <Param4 />
        <Param5 />
        <Sort>9</Sort>
        <Continue>1</Continue>
        <OsLimit>0</OsLimit>
        <Indentation>1</Indentation>
      </ScriptSteps>
    
  6. If you notice, the second ScriptSteps block has an ID specified in Param1, while the first one does not. This is where LT stripped out the non-existent EDF and shot itself in the foot. In my case, i just needed to find the EDF ID and replace <Param1 /> with <Param1>1234</Param1>. If you're not up to it, just delete the entire ScriptSteps block.
  7. Once done, save the file, and gzip it back up again:
    # gzip output
  8. Then, base64 encode the file:
    # base64 output.gz > fixed.txt
  9. Finally, copy the contents of fixed.txt and run an update to replace the ScriptData with the fixed version:
    UPDATE lt_scripts SET ScriptData='H4sI......AAA==' WHERE ScriptID=5986;
    (replace 5986 with your ScriptID, and paste the FULL content of what you selected into the ScriptData portion of the statement.)
  10. Run the query, and it will replace the body of the script with your backup copy.
  11. At this point you should be able to edit the script, no Refresh/Reload needed

Final Notes

  • Do not update the ScriptData of one script with the contents of another. The ScriptData has the GUID of the script encoded in it so it will still be broken.
  • Don't have access to a linux server, or don't know linux? Here are some alternatives:
    • For Base64 encoding/decoding, you can use your programming/scripting language of choice. Powershell has the ability to do Base64 Encoding/Decoding, a quick google can tell you how to do that. You can't really use websites for this portion because you are dealing with binary data.
    • For gzip compression, you can install 7zip to take care of this for you.
  • It's a good practice to export your scripts after making large changes. If you can store these in version control (such as git), that's even better.

Wednesday, April 19, 2017

Exchange 2013 IMAP Session Limitations

I have various services that require the use of IMAP to access mailboxes. In one case, a mailbox has subfolders and the service maintains an IMAP connection for each subfolder that it needs to connect to. The issue that came up was that after the 16th connection was opened, new connections were being rejected with no useful error (NO LOGIN FAILED or NO AUTHENTICATE FAILED). This seemed like an easy task, as there is an IMAP limit, so i ran:


Set-ImapSettings -MaxConnectionsPerUser 32 -Server FRONTENDSERVERNAME
Set-ImapSettings -MaxConnectionsPerUser 32 -Server BACKENDSERVERNAME

After restarting the IMAP service on both servers (or both services on the same server if you have all roles on one), i expected it to be fixed, and it wasn't. I checked the settings, confirmed it took effect, still nothing. After turning on protocol logging, the back-end server's logs indicated that the mailbox couldn't be opened due to Microsoft.Exchange.Data.Storage.TooManyObjectsOpenedException/Microsoft.Mapi.MapiExceptionSessionLimit. I have had to adjust a similar setting before for other reasons, but figured "it's not a MAPI connection, so why would that matter." Well as it turns out, the IMAP backend proxy just opens a MAPI connection to the mailbox, so yes it is needed.


To raise these limits you create a new DWORD value in HKLM\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem. After changing this, the Information Store service needs to be restarted. In my case, no one change actaully fixed the issue, it was a combination of the below values that worked:


  • Disable Session Limit - 0
  • Maximum Allowed Concurrent Exchange Sessions Per Service - 10000
  • Maximum Allowed Exchange Sessions Per Service - 10000
  • Maximum Allowed Service Sessions Per User - 10000
  • Maximum Allowed Sessions Per User - 10000

Probably a good idea to set these to something other than 10000, i'd gradually halve them until you run into issues again and then set it around there. It should also be noted that the IMAP service can take a little bit of time to stabilize, i wouldn't write off any of the changes until you've restarted the services and waited 10 minutes.

Friday, February 3, 2017

Xerox Scan to Folder via SMB with Windows Server 2016

Setting up a new Server 2016 file server, unable to get a Xerox WorkCentre 7220 to scan to it via SMB. This took a bit of troubleshooting, the copier was reporting invalid credentials and nothing was working. I had tried all combinations of IP vs Hostname, forward/backslashes, DOMAIN\username and username@domain, etc. There was a time mismatch between the server and copier, but after resolving that it was still not working. Ultimately i ended up running Wireshark on the server and found that the copier was trying to establish an SMB connection first to the IP address of the server, then to SMBSERVER (which is not the hostname of my server...).

Some googling came up with a few options, one of which would be to add SMBSERVER as an alternate NetBIOS name for the server. This included reducing some security levels and enabling some other things in the registry that didn't sound like a good idea. Instead, on a hunch, i switched the port from 139 (default) to 445. After switching the port, it worked as expected. Below are the settings that ended up working:

Server: SERVERNAME.domain.local : 445
Share: TheShareName
Document Path: Documents/Scans (no leading slash, and i used forward slashes)
Login Name: DOMAIN\username (we use a dedicated scans user)

See here:

Thursday, October 20, 2016

Microsoft Access and Outlook Integration - 48 Error in Loading DLL

A customer with a 3rd party developed Access database/application ran into this issue with some new computers. These were running 64 bit Windows 7 and 32 bit Office. The specific task was trying to open a new email in Outlook from Access.

While the error was 48 Error in loading DLL the problem had nothing to do with DLLs. It ended up being that because the computer was preloaded with Office 2016 (trial version, from Dell), when uninstalled it left some garbage behind in the registry. After running Procmon and finding out what it was trying to do, it wasn't too hard to find and fix.

The solution in the end was to delete the registry key (and subkeys):

HKEY_CLASSES_ROOT\TypeLib\{00062FFF-0000-0000-C000-000000000046}\9.6
Obviously make a backup of this key before deleting, but this did the trick for all affected computers.

Wednesday, August 17, 2016

PHP and imap_open with subfolders

This was tested against Exchange 2013, i assume similar environments will still apply. When using imap_open, if you're trying to access sub-folders there are a few caveats. Typically your connection string will look something like:
$imap=imap_open('{mail.server.com:143/imap/novalidate-cert}INBOX','username','password');

But there are exceptions to this:
  • To connect to a subfolder of the Inbox, the end of the connection string should read INBOX.subfoldername, note the dot instead of a slash.
  • To connect to a folder outside of the Inbox, the end should just be the folder name such as OtherFolder.
  • To connect to a subfolder of a folder outside of the Inbox, the end should read OtherFolder/subfoldername. Note that instead of using a dot, you actually need the slash here.