-->

Saturday, February 18, 2017

Exchange 2016 Room Mailbox Calendars Only Showing Busy

We recently migrated from Lotus Notes to Exchange 2016 and after moving Rooms and Equipment Mailboxes over, some of the calendars didn't keep their permissions.
When users went to schedule a room they wouldn't see who had booked the room, they would only see "Busy" like in this example:
 
Room Calendar No Details
 
This was strange because I left the default settings of AddOrganizerToSubject and DeleteSubject set to "True" so users should be able to see the organizer of the meeting.
 
After some digging around, I found that the Mailbox Folder Permissions for the "default" user were set  to "none"...but not on every Room mailbox. Looks like the migrator tool wasn't able to keep those permissions intact.
 
To view permissions on a Mailbox fire up the Exchange Management Shell and run:

Get-MailboxFolderPermission "room 1:\Calendar"

**Note** Change "room 1" to the room mailbox name in your environment.

In my case permissions were like so:

FolderName     : Calendar
User                 : Default
AccessRights   : {None}
Identity            : Default
IsValid             : True

Luckily with PowerShell, this is an easy fix. We're gonna set permissions on all Room Mailboxes, so we don't have to go digging through each of them.

**Note** You might get a bunch of yellow warnings saying no changes were made on some Mailboxes; that's ok, it just means the perms were already set.
 
In the (EMS) and run the following cmdlets:
 
$r = Get-Mailbox -RecipientTypeDetails RoomMailbox
 
$r | %{Set-MailboxFolderPermission $_":\Calendar" -User Default -AccessRights LimitedDetails}
 
**Note** I chose "LimitedDetails" because I don't want users seeing everything about meetings in case they might be sensitive like "we're closing down your office". You can set different levels of access, which are listed in this TechNet article.

Now open a Room Calendar in Outlook or OWA, and you should see the Organizer and Room:

Room Calendar With Details

 Now you'll have happy users!

Sunday, February 5, 2017

Exchange Veeam Backup Timeout - VSSControl: Failed to prepare guest for freeze timeout 900 sec

We're using the Quest tool to migrate a ton of mailboxes from Notes over to Exchange, and since it uses quite a bit of resources on the Exchange servers, our Veeam backups were failing. This led to a massive amount of transaction logs building up in Exchange, which led to more backup failures. Round and round we went, never getting a good backup.

The error in Veeam was:

“VSSControl: Failed to prepare guest for freeze, wait timeout 900 sec."

The issue was that Veeam was timing out because it couldn't freeze the guest VM (the Exchange mailbox server) in order to start the backup. After the 900 seconds (15 minutes) expired, Veeam would give up and fail.

What I had to fix it was adjust a registry setting on the Veeam server, to lengthen the timeout threshold.

On the Veeam server, open regedit and navigate to:

HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication

Add a DWORD (32-bit) named VssPreparationTimeout

**Note** The value is in milliseconds (decimal), the default timeout is 900000, which equals 15 minutes

We're going to start by setting it to 30 minutes, so set the value to 1800000

**Note** Even after 30 minutes, mine kept failing so I had to keep bumping it up in increments until I found the sweetspot...which for me was 66 minutes.

After you create the DWORD, close regedit.

Stop any running jobs or wait for them to complete (which they won't), close the Veeam control panel, then restart the Veeam Backup Service.

Open the Veeam control panel up, and start your backup job.

Now it should freeze the guest successfully, and get a good backup!

Exchange/Active Directory Add Proxy Addresses in Bulk

In my current project, we're migrating from Lotus Notes to Exchange 2016 in a Resource Forest, and we needed to do some manual mail routing. That involved setting proxy addresses in our Accounts Forest.
 
In my previous post I went over how to set the targetAddress with ADModify.
 
This time we'll be adding entries to the proxyaddresses by using PowerShell.
 
I used two methods of bulk adding proxies; by Organizational Unit (OU) and by importing a CSV (our OU structure is a mess, and we had some users scattered about so I used a CSV list to finish up).
 
We're going to add a proxy address to the already existing ones.
This new proxy will be an accepted domain in our Exchange environment, but not the authoritative one. We'll use something like @test.domain.com. We basically want to be able to route mail from Notes to Exchange, using a temporary address.

My organization's email address is the common format of firstname.lastname@domain.com, so the cmdlet will pull the GivenName (first name) and Surname (last name) of each user, and stick a "." in the middle, while adding the test accepted domain to the end.

To bulk add proxies by OU, fire up the Active Directory PowerShell, and run:

Get-ADUser -Filter 'Name -like "*"' -SearchBase 'OU=US,DC=exchangeitup,DC=com' -Properties proxyaddresses | % {Set-ADUser $_ -add
@{proxyAddresses="smtp:"+ $_.GivenName + '.' + $_.Surname +"@test.exchangeitup.com"}}

**Note** You'll want to change 'OU=US,DC=exchangeitup,DC=com' and test.exchangeitup.com" to match your environment.

**Note** You'll also notice that the "smtp:" is lowercase; that is by design because we want it as an alias, not primary address.

After the cmdlet runs, check over a few users AD attributes and you should see the newly added proxyaddresses entries.

Now, we'll see how to import a CSV list of users and set the proxies.

First, create a CSV in the following format.

SAM,SMTP
username1,user1@test.exchangeitup.com
username2,user2@test.exchangeitup.com
username3,user3@test.exchangeitup.com
username4,user4@test.exchangeitup.com

You'll have one column, titled SAM, and one titled SMTP; with the usernames (samaccountnames) and accepted domain email addresses listed.

Save it to wherever you're going to run AD PowerShell from, and name it something like proxies.csv - specific, I know :)

Now, fire up the AD PowerShell and run the following:

import-csv proxies.csv | foreach {Get-ADUser $_.SAM | Set-ADUser -add @{proxyaddresses = "smtp:"+($_.smtp)}}

This cmdlet will add the proxy addresses (as an alias with the lowercase "smtp:") using the samname and the other accepted domain (test.exchangeitup.com) we'll be using for our routing.

Once again, go check a couple users' AD attributes and you'll see the newly added proxies.

Happy migrating!

Friday, February 3, 2017

Exchange 2016 100% CPU and Event ID 5011 "MSExchangeServicesAppPool"

This post is mostly for my own reference, but just in case you come across the same issue, I'll outline the problem and the fix.

We recently migrated several hundred users from Notes to my Exchange 2016 CU2 environment and shortly after the migrations, one of my three mailbox servers was getting constantly hammered.

The CPU was at 100% usage, performance was terrible, and it even caused my Kemp load balancer to flap the virtual services on this server every 30 seconds or so. In particular the MAPI and EWS services.

That led me to start checking what was up with EWS.

The System Event log was full of the following error:

Log Name:      System
Source:        Microsoft-Windows-WAS
Date:          1/28/2017 3:11:50 PM
Event ID:      5011
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      EXCH16-MBX1
Description:
A process serving application pool 'MSExchangeServicesAppPool' suffered a fatal communication error with the Windows Process Activation Service. The process id was '18076'. The data field contains the error number.


I remembered that one of our admins was testing out an EWS Retention policy script and had installed the Exchange Web Services Managed API 2.2  on this box.

So, I uninstalled the API 2.2 and bingo! Problem solved; CPU was back to normal, all client connections were now stable.

As a workaround, we installed the EWS Managed API 2.2 on our DAG witness server (since it has the Exchange tools installed on it) and now run the script from there.

It works!

Saturday, January 28, 2017

Exchange Set TargetAddress Attribute in Bulk With ADModify

My current project involves migrating from Lotus Notes to Exchange 2016, and since we didn't pay for a full suite of a certain migration utility, we had to do some mail forwarding the old fashioned way: setting the targetAddress attribute on the AD accounts.
The targetAddress is basically a forwarding address that routes mail to a different mail environment...in this case from Notes to Exchange.

In our cutover, I had to set the target for 800 users in bulk, so I chose to use AdModify. You could write a PowerShell script to do this. but why reinvent the wheel? :)

This article assumes you already have ADModify downloaded and know how to use it.
If not, here's an overview.

We're going to set the 'targetAddress' to same as the 'mail' attribute; this will forward our mail to the proper location - from Notes to Exchange.

Once you have ADModify loaded up and you have your OU's and users specified, navigate to the Custom Tab:

ADModify Custom Tab
 
 
Put a check next to "Make a customized attribute modification"
 
In the Attribute Name, enter: targetAddress
In the Attribute Value enter: %'mail'%
 
This will set the target to same value as mail. You can use other attributes, like mailnickname if you have that populated.
 
**Note** You  must use the percent sign % and apostrophe ' at each end of the mail attribute name.
 
Next, hit the Go! button.
 
ADModify targetAddress to mail
 
 
Review successes and errors.
 
Now you can check ADUC (Active Directory Users and Computers) on one or a few of the accounts you modified.
 
Open Properties of the account, navigate to the Attributes Tab and scroll down to "targetaddress". The new address matching the "mail" address should be listed.
 
Happy migrating!

Saturday, January 14, 2017

Exchange Search-Mailbox Delete More Than 10,000 Items

A few days ago one of my buddies (who happens to be running the Exchange 2013 environment at my old job) called and had a problem where a user got spammed with 13,000,000 messages. Yes, you read that correctly, that's 13 million!
Normally, you would run the Search-Mailbox cmdlet with the -DeleteContent switch to clear those out.
The problem here is, the Search-Mailbox command is limited to only 10,000 messages, and you would have re-run it until you clear out the mailbox...with 13 million, you'd be running that cmdlet a ton of times!

So I created a quick and dirty script that will loop the Search-Mailbox until it doesn't find any more instances of the message.

Copy and paste the following code into notepad and save it as a .ps1 file. For instance Delete-HugeSpam.ps1

$mbx = get-mailbox "mailboxname here"

Do {
 $result = Search-Mailbox -Identity $mbx.Identity -SearchQuery 'Subject:"this is spam from a dirty spammer"' -DeleteContent -force -WarningAction Silentlycontinue

 
write-host $result.resultitemscount -ForegroundColor Green

 } Until ($result.resultitemscount -eq 0)


**Note** Change "mailboxname here" to the mailbox with all the spam, and "this is spam from a dirty spammer" to whatever the Subject of the message is.

Once you have the .ps1 file configured with your mailbox name and subject, fire up the Exchange Management Shell (EMS),  and cd to the directory where you saved the .ps1, then run:

Delete-HugeSpam.ps1

You can monitor the mailbox from OWA by giving yourself FullAccess. I wouldn't recommend using Outlook if there's thousands or millions (in this case) of items because it'll prolly never open.
You should then see the message count doing down.

Now, instruct your user not to open suspicious emails, or better yet don't allow them to have a mailbox anymore :)

Saturday, January 7, 2017

Exchange/Outlook Not Showing Quota Information in Status Bar

In my Exchange environment, we have a requirement to show user mailbox quota information on the Status Bar in Outlook. This is actually one of the better requirements I've dealt with, so users can monitor their mailbox size themselves :)
 
The problem I had was: when enabling the quota information in Outlook, it didn't actually turn on.
 
Outlook Quota Off
 
As you can see, Quota Information is checked, but it still shows "Off"
 
The reason is, you have to set the Send and Receive Limit in Exchange to an actual value...it can't be set to "unlimited".
 
What I mean by that is: I usually just set the Issue Warning and Prohibit Send settings to a value like 50GB and leave Send/Receive set as unlimited, because I want users to be able to receive mail even if they bump into their quota limit, but not be able to send mail until they clean it out - this results in less helpdesk calls that users aren't getting messages.
 
For the quota info to show in Outlook, you have to set all three limits:
 
For Outlook clients in cached mode, you need to set the following in Exchange:

Issue warning
Prohibit send
Prohibit send and receive


For Outlook clients in online mode, you need to set the following in Exchange:

Prohibit send
Prohibit send and receive


Unless you control Outlook connections with a GPO, you probably have a mix of online and cached clients, so you'll need to set all three limits (warning, prohibit send, and prohibit send/receive)

To quickly set these on all Databases fire up the Exchange Management Shell (EMS) and run the following cmdlet:

Get-MailboxDatabase | Set-MailboxDatabase -IssueWarningQuota 45GB -ProhibitSendQuota 50GB -ProhibitSendReceiveQuota 100GB

**Note** Change the limits to how you see fit. I'd advise you to set "-ProhibitSendReceiveQuota" to something pretty high, to keep it close to the unlimited setting (if you had that set previously).

You can also use the EAC to change those limits, but depending on how many databases you have, that could be a waste of time.

To do so, navigate to servers > databases.

Select a DB and click the Edit (Pencil button) and then click limits.

Enter the values for three I mentioned earlier and click save:

Exchange DB limits
 

For users to enable the quota information themselves; in Outlook, right-click the Status Bar (at the bottom of the window) and select "Quota Information".

This will place a checkmark next to the setting and it will switch to "On":

Outlook Quota On

Now you see that the "On" flag is set and the quota is shown in the bottom-left corner...yes that's my mailbox, and I keep very little email - contrary to popular belief, its not a repository :)

Create a GPO to push the Quota Information Setting to clients:

If you want to force the setting to "on" for all outlook clients, you'll need to push a GPO out.

Open the Group Policy Management Console > Create a new GPO > link it to an OU > Give it a name like Outlook_2016_Quota

**Note** You can also edit an existing GPO if you have one set already.

Scroll down to User Configuration > Preferences > Windows Settings > Registry

Right-Click > New > Registry Item

Set the following values:

Action - Replace
Hive - HKEY_CURRENT_USER
Key Path - SOFTWARE\Microsoft\Office\16.0\Outlook\StatusBar
Value Name - QuotaThermometer
Value Type - REG_DWORD
Value Data - 00000001
Base - Hexadecimal

**Note** If you have a mix of outlook versions, replace the \16.0\ with the version number.

On a client machine that's part of the OU you specified, fire up an Elevated CMD and run:

gpupdate /force

Start Outlook and you should see the Quota info being shown.

You can also check in the registry on the client machine under: HKCU\SOFTWARE\Microsoft\Office\16.0\Outlook\StatusBar and you'll see the "QuotaThermometer" key:

Outlook QuotaThermometer Reg Key
 
Now your users will hopefully manage their mailbox sizes more efficiently!

Wednesday, December 21, 2016

Exchange 2016 Export Mailbox Error: "Couldn't find the Enterprise Organization container"

I've been running Exchange 2016 environment for a little while now, but I finally had to export a mailbox to PST.

I kept getting an error in both the EAC (Exchange Admin Console) and the Shell.

The error:

New-MailboxExportRequest -Mailbox "staceybranham" -FilePath \\server1\PSTs\stacey.pst

Couldn't find the Enterprise Organization container

This error is usually thrown when you haven't added your admin account to be allowed to import/export mailboxes.

The "funny" thing is: I had already run the following cmdlet to do so:

New-ManagementRoleAssignment -Role "Mailbox Import Export" -User Admin

But, every time I ran the export it would error out.

What I forgot to do, was close and reopen both the EMS (Exchange Management Shell) and the EAC.

After that, I was able to run the export without the error.
By the way this is a stupid error since the Enterprise Organization Container obviously exists, or there'd be no Exchange.

How about next time just have it say "you don't have a Management Role set for this account" or something to that effect? It would save a lot of digging around to ensure that the containers are set correctly...ugh

Saturday, December 10, 2016

Kemp Load Balancer: Change from One-Arm to a Two-Arm Configuration

During the planning and set up phase of my current Exchange 2016 project, management stipulated that no external Exchange access was to be allowed - less pain for me!
Several months later after the environment is in production, they now want to allow external access...go figure :)

There were several issues here: we have a one-arm Kemp virtual load balancer installed on the internal LAN; the firewall/security team didn't want to NAT anything internally (they will only NAT to the DMZ); NATing to the DMZ requires another load balancer (costing more money) or a reverse proxy solution, which would then have to redirect back to the internal Kemp...not a graceful solution.

I decided that it would save a ton of trouble just switching the existing Kemp from a one-arm config to a two-arm.
Meaning, it will have one NIC in the internal LAN, and one NIC in the DMZ thereby allowing both internal and external client connections.

The steps are pretty straightforward and some of the things will be specific to your environment.

**Disclaimer** You should not perform this on a production environment as it can result in loss of client connections...but if you have no choice, go for it!

**Note** You should take a back up the current config before starting the changes.

**Note** You will have to restart the load balancer during the changes, so it's best to do this outside business hours.

Changing a One-Arm Kemp LoadMaster to a Two-Arm Configuration:

First, you'll need to add a second NIC to your load balancer VM and assign it to another network, such as your DMZ.

In my case, I already had two NICs defined (eth0 and eth1) but eth1 wasn't being used, so I set the DMZ network on that one.

My initial setup was eth0 on the internal 10.x.69.x network, which allowed only internal connections.

Kemp eth0

Now, I've added the DMZ network to eth1, which is on the 10.x.68.x network.

Kemp eth1

In our network, the DMZ isn't routable back to internal, so we need to do some extra configuration on the Kemp itself.

We'll need to ensure that we don't have asymmetric back end routes, which occurs in two-arm setups, by enabling "Subnet Originating Requests". We'll also need to "Enable Non-local real Servers".

Both of the above settings are found in Systems Configuration > Miscellaneous Options > Network Options.

Put a checkmark for both Subnet Originating Requests and Enable Non-local real Servers
 
Kemp Enable Non-local and Subnet Originating

Next, we'll need to configure the WUI (Web User Interface) to be reachable from the internal NIC.

Navigate to Certificates > Security > Remote Access

Here, next to "Allow Web Admin Access" we'll use the drop-down box to select our Internal NIC, which on my system is eth0.

We'll then set the "Admin Default Gateway" to our Internal LAN Gateway on the 10.x.69.x subnet; this ensure we can still get to the WUI once we change our routes later.

**Note** Immediately after you change the Admin Default Gateway, you will lose WUI access, and the next step must be from the console.

Kemp WUI Admin Access

Next we'll need to set the Default Gateway to the DMZ network, since our DMZ is not routable to the internal LAN.


**Note** This step requires console access because since we changed the admin gateway we lost WUI access. While in console, you will also need to reboot your device after changing the gateway.

**Note** The Default Gateway is a global setting, there isn't an option for setting it on each NIC.

In the console, login to your LoadMaster, and navigate to (4) Basic Setup.

Kemp CLI Basic Setup

Next, we'll choose option (4) Default Gateway Configuration.

Kemp CLI Default GW

Here, we'll set the GW to your DMZ Gateway, in my case this is my 10.x.68.x subnet then TAB down to OK and hit Enter when finished.
 
Kemp CLI Set GW

Next select (q) return to previous menu. Back at the main menu select (8) Reboot.

Kemp CLI Reboot

Now, the last step is creating our new Virtual Services (VS) and Virtual IP Address (VIP) on the DMZ interface. This is done exactly like you configured your internal VIP and VS, by using the Exchange templates and existing Exchange cert.

Go to Virtual Services > Add New

Assign your Virtual Address (VIP) which will be on your DMZ subnet.

Give it a Service Name - this is not optional.

I added the word "external" to the service name that auto-populates: when adding another VS using the same template, because it will automatically try to use the same service name as the already configured internal VS and they cannot be the same.

In the Use Template drop-down, select Exchange 2016 HTTPS Reencrypted...the port will magically change to 443.

Click "Add this Virtual Service"

Kemp Add New Virtual Service

Go to Virtual Services > View/Modify Services, you should now see your second VS on the DMZ VIP.

In my example, my DMZ Virtual Services are on the 10.28.68.135 VIP and my Internal VS are on the 10.28.69.167 VIP.

Kemp Internal/External Services

Now we need to configure the new DMZ SubVS just like you did for you internal Virtual Services, and then assign your Exchange Certificate.

If you need a refresher on how to configure those SubVS and certs, follow Gareth Gudger's (SuperTekBoy) awesome post here; you'll just be doing the same steps but this time it'll be on the newly created DMZ Virtual Services.

Once you finish those settings, you're ready to test!

From an external machine try to hit your OWA, set up Outlook, and ActiveSync on your device...if your NATing and firewall rules are set correctly you should have no problems with external access.

Now to get less sleep since you can check email from anywhere!

Saturday, December 3, 2016

Exchange 2016 Management Shell Won't Connect Error: The WinRM Shell client cannot process the request

The other day after patching and bouncing my Exchange 2016 servers, one of them came back up pretty grumpy.
All services were running, and mailflow was healthy, but the Exchange Management Shell (EMS) threw WinRm errors when opening.
The Application and System Event Logs were full of warnings and errors too.

Luckily the event logs are little more helpful, because the error thrown in the EMS didn't really tell me what was actually going on.

EMS Error:

Connecting to EXCHMBX01.exchangeitup.com.
 New-PSSession : [EXCHMBX01.exchangeitup.com] Processing data from remote server EXCHMBX01.exchangeitup.com failed
 with the following error message: The WinRM Shell client cannot process the request. The shell handle passed to the WSMan Shell function is not valid. The shell handle is valid only when WSManCreateShell function completes successfully. Change the request including a valid shell handle and try again. For more information, see the about_Remote_Troubleshooting Help topic.


As you can see, processing data failed...WinRM...ok what does that mean? After it throws the error, the Shell will remote session over to another Exchange server, so at least you're not dead in the water.

Event Logs:

Log Name:      Application
Source:        MSExchange Front End HTTP Proxy
Date:          11/28/2016 10:16:41 PM
Event ID:      1003
Task Category: Core
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      EXCHMBX01
Description:
[PowerShell] An internal server error occurred. The unhandled exception was: System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Exchange.HttpProxy.RemotePowerShellProxyRequestHandler.ExposeExceptionToClientResponse(Exception ex)
   at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CompleteWithError(Exception ex, String label)
   at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<>c__DisplayClass29.b__28()
   at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)


Log Name:      Application
Source:        ASP.NET 4.0.30319.0
Date:          11/28/2016 10:16:40 PM
Event ID:      1309
Task Category: Web Event
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      EXCHMBX01
Description:
Event code: 3005
Event message: An unhandled exception has occurred.
Event time: 11/28/2016 10:16:40 PM
Event time (UTC): 11/29/2016 3:16:40 AM
Event ID: 90a2cfed15c84e1ab33affbe5cb62d96
Event sequence: 2
Event occurrence: 1
Event detail code: 0

Application information:
    Application domain: /LM/W3SVC/1/ROOT/PowerShell-1-131248629080807714
    Trust level: Full
    Application Virtual Path: /PowerShell
    Application Path: C:\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy\PowerShell\
    Machine name: EXCHMBX01

Process information:
    Process ID: 9212
    Process name: w3wp.exe
    Account name: NT AUTHORITY\SYSTEM

Exception information:
    Exception type: NullReferenceException
    Exception message: Object reference not set to an instance of an object.
   at Microsoft.Exchange.HttpProxy.RemotePowerShellProxyRequestHandler.ExposeExceptionToClientResponse(Exception ex)
   at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CompleteWithError(Exception ex, String label)
   at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<>c__DisplayClass29.b__28()
   at Microsoft.Exchange.Common.IL.ILUtil.DoTryFilterCatch(Action tryDelegate, Func`2 filterDelegate, Action`1 catchDelegate)
   at Microsoft.Exchange.HttpProxy.Diagnostics.SendWatsonReportOnUnhandledException(Action methodDelegate, LastChanceExceptionHandler exceptionHandler)
   at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.CallThreadEntranceMethod(Action method)


Log Name:      System
Source:        Microsoft-Windows-HttpEvent
Date:          11/28/2016 10:25:39 PM
Event ID:      15021
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      EXCHMBX01
Description:
An error occurred while using SSL configuration for endpoint 0.0.0.0:444.  The error status code is contained within the returned data.

From the above logs, we finally see that there's something wrong with the HTTP/SSL config on the PowerShell virtual directory.

The Fix:

Close the Exchange Management Shell.

Open IIS Manager, and drill down to Servername > Sites. Right-click Exchange Back End and select Edit Bindings.

IIS Edit Bindings

In the Site Binding window, select the HTTPS binding, and click Edit.

Under SSL Certificate, it will say Not Selected - meaning there's no certificate assigned to the HTTPS service on port 444, which was referenced in the System log Event 15021.

Click the drop-down and select your applicable certificate - this will usually the self-signed cert, but you can check your other working servers to see which one is assigned.

IIS Assign Cert

Click OK

Fire up the EMS and you should have a successful connection!

Saturday, November 26, 2016

Exchange 2013/2016 Cleaning up old Diagnostic and IIS Log Files

A while back I cross-posted an article by Jason Sherry on cleaning up Exchange 2013 Log files with a one-liner.
Jason's solution worked very well, until I stood up a new Exchange 2016 environment, and it just wouldn't run - no errors, it just didn't do anything.
So, I came up with a scheduled task that works awesome, and keeps those pesky logs file down to a manageable size.

A little back story:
Exchange 2013/2016 keeps pretty much every diagnostic log in order for MS Support to easily go back and find problems without having to wait for Exchange admins to manually pull logs. The problem is: Exchange does a very poor job of cleaning up said logs and they fill up your system volume very quickly.

I know, I know, disks are cheap - well businesses are cheap too, and won't always pay for what you need!

Creating the "Clean Logs" Scheduled Task:

The following task will clean up the Exchange Daily Performance Logs and IIS logs keeping only the past 3 days.

Create a Service Account:

First, you'll want to create a Service Account in your domain, which will be used to run the scheduled task. It's best practice to use service accounts rather than your own account to run scheduled tasks, so if you ever leave your position and they deactivate your account, it won't break the task!

In your domain, create a new user called something like exchscriptrunner and set a super-strong password.
The only membership it needs is Domain User, so it should be good to go.

Next, add the newly created user to the Local Administrators Group on each Exchange server. The scheduled task will need local admin rights to run PowerShell things, and since you have a super strong password, it's not an issue.

Create the scheduled task on the first Exchange server:
On one of the Exchange servers in the Task Scheduler Control Panel, click Action > Create Task...


On the General tab:

Give it a name like Clean Logs

Click "Change User or Group..." hit "Locations" and switch to your domain, then search for your exchscriptrunner service account.

Check the box for "Run with highest privileges"



On the Triggers Tab:

Click "New..."
Set it for how often you need it to clean up the logs. I run mine Daily at 10AM - no specific reason, but you do want it to run Daily. You can run It during production hours as it doesn't use many resources.

 


On the Actions Tab:

Set the "Action" dropdown to "Start a program"

Under Program/Script, copy/paste the following:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

In the "Add arguments" field, copy/paste the following:

-NonInteractive -WindowStyle Hidden -command ". 'C:\Program Files\Microsoft\Exchange Server\V15\bin\RemoteExchange.ps1'; Connect-ExchangeServer -auto; Get-ChildItem 'C:\Program Files\Microsoft\Exchange Server\V15\Logging','C:\Inetpub\Logs' -Directory | Get-ChildItem -Include '*.log','*.blg' -Recurse | ? LastWriteTime -lt (Get-Date).AddDays(-3) | Remove-Item -ErrorAction silentlycontinue

**Note** This assumes that you have installed Exchange to the default location. If you have installed it to another location, change the "C:\Program Files\Microsoft\Exchange Server\V15\bin\RemoteExchange.ps1" and "C:\Program Files\Microsoft\Exchange Server\V15\Logging','C:\Inetpub\Logs" to match your environment.

**Note** This keeps the last 3 days of logs, which should be sufficient for any error intel gathering...if 3 days have gone by and you haven't noticed problems in Exchange, you might need some monitoring :)
To change it to more or less than 3 days, set the "AddDays(-3)" to a different number value.

 


In the Settings Tab:

Checkmark the following boxes:

- Allow task to be run on demand

- Stop the task if it runs longer than: 1 hour (if it runs longer than an hour, you got something wrong!)

- If the running task does not end when requested, force it to stop

 



Click OK when you have everything set.

Testing the Clean Logs Task:

In the main task window, right-click your new "Clean Logs" task, and click Run.

When it finishes running, you should have a (0x1) Last Run Result.

You can also go check C:\Program Files\Microsoft\Exchange Server\V15\Logging\Diagnostics\DailyPerformanceLogs (or wherever you've installed Exchange) and you should see only three days' worth of log files.

Create the scheduled task on the other Exchange servers:


Once you've successfully run the task, you'll need to create it on your other Exchange servers...this is super easy!

In the main Task Scheduler window, right-click your "Clean logs" task and click "Export" and save the file to your preferred location.

This will create an .xml file with all the settings intact.

Copy the Clean logs.xml file to each of your other Exchange servers. 


On each Exchange server, open the Task Scheduler.

Click the Task Scheduler Library folder in the left pane.

Click Action, and "Import Task..."

Select your Clean logs.xml file and hit Ok.

Now you'll see the Clean logs task at the ready state in the Task Library on each server.

Done!

To supplement the scheduled task, if you have rather small Exchange install volumes, I also like to compress the C:\Program Files\Microsoft\Exchange Server\V15\Logging directory - with zero adverse effects. To do that,
follow my post here.

Now, with this scheduled task and a compressed logging directory, you should save tons of space on your servers, and be free from filling up the drive causing mailflow to fail ;)