Collaborate for free with online versions of Microsoft Word, PowerPoint, Excel, and OneNote. Save documents, spreadsheets, and presentations online, in OneDrive. Download Microsoft Outlook for macOS 10.13 or later and enjoy it on your Mac. A qualifying Microsoft 365 subscription is required for this application. Microsoft 365 includes premium Office apps, extra cloud storage in OneDrive, advanced security, and more, all in one convenient subscription. Sep 22, 2020.
Do you need to know the steps to take to identify and fix lags, hangs, and slow performance between SharePoint Online, OneDrive for Business, Exchange Online, or Skype for Business Online, and your client computer? Before you call support, this article can help you troubleshoot Office 365 performance issues and even fix some of the most common issues.
This article is actually a sample action plan that you can use to capture valuable data about your performance issue as it's happening. Some top issues are also included in this article.
If you're new to network performance and want to make a long term plan to monitor performance between your client machines and Office 365, take a look at Office 365 performance tuning and troubleshooting - Admin and IT Pro.
Sample performance troubleshooting action plan
This action plan contains two parts; a preparation phase, and a logging phase. If you have a performance problem right now, and you need to do data collection, you can start using this plan right away.
Prepare the client computer
Log the performance issue
If you're not familiar with running the tools mentioned in this article, don't worry because we provide those steps next. If you're accustomed to doing this kind of network capturing, you can skip to How to collect baselines, which describes filtering and reading the logs.
Flush the DNS Cache first
Why? By flushing out the DNS cache you're starting your tests with a clean slate. By clearing the cache, you're resetting the DNS resolver contents to the most up-to-date entries. Remember that a flush does not remove HOSTs file entries. If you use HOST file entries extensively, you should copy those entries out to a file in another directory and then empty the HOST file.
Flush your DNS resolver cache
Netmon
Microsoft's Network Monitoring tool (Netmon) analyzes packets, that is traffic, that passes between computers on networks. By using Netmon to trace traffic with Office 365 you can capture, view, and read packet headers, identify intervening devices, check important settings on network hardware, look for dropped packets, and follow the flow of traffic between computers on your corporate network and Office 365. Because the actual body of the traffic is encrypted, that is, it(travels on port 443 via SSL/TLS, you can't read the files being sent. Instead, you get an unfiltered trace of the path that the packet takes which can help you track down the problem behavior.
Be sure you don't apply a filter at this time. Instead, run through the steps and demonstrate the problem before stopping the trace and saving.
After you install Netmon 3.4, open the tool and take these steps:
Take a Netmon trace and reproduce the issue
HTTPWatch
HTTPWatch comes in charged, and a free edition. The free Basic Edition covers everything you need for this test. HTTPWatch monitors network traffic and page load time right from your browser window. HTTPWatch is a plug-in to Internet Explorer that graphically describes performance. The analysis can be saved and viewed in HTTPWatch Studio.
Note
If you use another browser, such as Firefox, Google Chrome, or if you can't install HTTPWatch in Internet Explorer, open a new browser window and press F12 on your keyboard. You should see the Developer Tool pop-up at the bottom of your browser. If you use Opera, press CTRL+SHIFT+I for Web Inspector, then click the Network tab and complete the testing outlined below. The information will be slightly different, but load times will still be displayed in milliseconds. > HTTPWatch is also very useful for issues with SharePoint Online page load times.
Run HTTPWatch and reproduce the issue
HTTPWatch is a browser plug-in, so exposing the tool in the browser is slightly different for each version of Internet Explorer. Typically, you can find HTTPWatch under the Commands bar in the Internet Explorer browser. If you don't see the HTTPWatch plug-in in your browser window, check the version of your browser by clicking Help > About, or in later versions of Internet Explorer, click the gear symbol and About Internet Explorer. To launch the Commands bar, right-click the menu bar in Internet Explorer and click Commands bar.
In the past, HTTPWatch has been associated with both the Commands and the Explorer bars, so once you install, if you don't immediately see the icon (even after reboot) check Tools, and your toolbars for the icon. Remember that toolbars can be customized and options can be added to them.
This screen shot is from the Professional version of HTTPWatch. You can open traces taken in the Basic Version on a computer with a Professional version and read it there. Extra information may be available from the trace through that method.
Problem Steps Recorder
Steps Recorder, or PSR.exe, allows you to record issues as they are occurring. It's a very useful tool and very simple to run.
Run Problem Steps Recorder (PSR.exe) to record your work
The date and time is recorded for you. This links your PSR to your Netmon trace and HTTPWatch in time, and helps with precision troubleshooting. The date and time in the PSR record can show that a minute passed between the login and browsing of the URL and the partial render of the admin site, for example.
Read your traces
It isn't possible to teach everything about network and performance troubleshooting that someone would need to know via an article. Getting good at performance takes experience, and knowledge of how your network works and usually performs. But it is possible to round up a list of top issues and show how tools can make it easier for you to eliminate the most common problems.
If you want to pick up skills reading network traces for your Office 365 sites, there is no better teacher than creating traces of page loads regularly and gaining experience reading them. For example, when you have a chance, load an Office 365 service and trace the process. Filter the trace for DNS traffic, or search the FrameData for the name of the service you browsed. Scan the trace to get an idea of the steps that occur when the service loads. This will help you learn what normal page load should look like, and in the case of troubleshooting, particularly around performance, comparing good to bad traces can teach you a lot.
Netmon uses Microsoft Intellisense in the Display filter field. Intellisense, or intelligent code completion, is that trick where you type in a period and all available options are displayed in a drop-down selection box. If, for example, you are worried about TCP window scaling, you can find your way to a filter (such as
.protocol.tcp.window < 100 ) by this means.
Netmon traces can have a lot of traffic in them. If you aren't experienced with reading them, it's likely you will be overwhelmed opening the trace the first time. The first thing to do is separate the signal from the background noise in the trace. You tested against Office 365, and that's the traffic you want to see. If you are used to navigating through traces, you may not need this list.
Traffic between your client and Office 365 travels via TLS, which means that the body of the traffic will be encrypted and not readable in a generic Netmon trace. Your performance analysis doesn't need to know the specifics of the information in the packet. It is, however, very interested in packet headers and the information that they contain.
Tips to get a good trace
If you're only using Netmon tracing at the time of the problem, that's okay too. To orient yourself, use a filter like
ContainsBin(FrameData, ASCII, 'office') or ContainsBin(FrameData, ASCII, 'outlook') . You can record your frame number from the trace file. You may also want to scroll the Frame Summary pane all the way to the right and look for the Conversation ID column. There is a number indicated there for the ID of this specific conversation that you can also record and look at in isolation later. Remember to remove this filter before applying any other filtering.
Tip
Netmon has a lot of helpful built-in filters. Try the Load Filter button at the top of the Display filter pane.
Get familiar with your traffic, and learn to locate the information you need. For example, learn to determine which packet in the trace has the first reference to the Office 365 service you're using (like 'Outlook').
Taking Office 365 Outlook Online as an example, the traffic begins something like this:
![]()
All parts of the traffic are important and connected, but small portions of the trace contain information particularly important in terms of performance troubleshooting, so we'll focus on those areas. Also, since we've done enough Office 365 performance troubleshooting at Microsoft to compile a Top Ten list of common problems, we'll focus on those issues and how to use the tools we have to root them out next.
If you haven't installed them all ready, the matrix below makes use of several tools. Where possible. Links are provided to the installation points. The list includes common network tracing tools like Netmon and Wireshark, but use any tracing tool you are comfortable with, and in which you're accustomed to filtering network traffic. When you're testing, remember:
Common issues
Some common issues you may face and how to find them in your Network trace.
TCP Windows Scaling
Found in the SYN - SYN/ACK. Legacy or aging hardware may not take advantage of TCP windows scaling. Without proper TCP windows scaling settings, the default 16-bit buffer in TCP headers fills in milliseconds. Traffic cannot continue to send until the client receives an acknowledgment that the original data has been received, causing delays.
Tools
What to look for
Look for the SYN - SYN/ACK traffic in your network trace. In Netmon, use a filter like
tcp.flags.syn 1 . This filter is the same in Wireshark.
Notice that for every SYN there is a source port (SrcPort) number that is matched in the destination port (DstPort) of the related Acknowledgment (SYN/ACK).
To see the Windows Scaling value that is used by your network connection, expand first the SYN, and then the related SYN/ACK.
TCP Idle Time Settings
Historically, most perimeter networks are configured for transient connections, meaning idle connections are generally terminated. Idle TCP sessions can be terminated by proxies and firewalls at greater than 100 to 300 seconds. This is problematic for Outlook Online because it creates and uses long-term connections, whether they are idle or not.
When connections are terminated by proxy or firewall devices, the client is not informed, and an attempt to use Outlook Online will mean a client computer will try, repeatedly, to revive the connection before making a new one. You may see hangs in the product, prompts, or slow performance on page load.
Tools
What to look for
In Netmon, look at the Time Offset field for a round-trip. A round-trip is the time between client sending a request to the server and receiving a response back. Check between the Client and the egress point (ex. Client --> Proxy), or the Client to Office 365 (Client --> Office 365). You can see this in many types of packets.
As an example, the filter in Netmon may look like
.Protocol.IPv4.Address 10.102.14.112 AND .Protocol.IPv4.Address 10.201.114.12 , or, in Wireshark, ip.addr 10.102.14.112 && ip.addr 10.201.114.12 .
Tip
Don't know if the IP address in your trace belongs to your DNS server? Try looking it up at the command line. Click Start > Run > and type cmd, or press Windows Key > and type cmd. At the prompt, type
nslookup <the IP address from the network trace> . To test, use nslookup against your own computer's IP address. > To see a list of Microsoft's IP ranges, see Office 365 URLs and IP address ranges.
If there is a problem, expect long Time Offsets to appear, in this case (Outlook Online), particularly in TLS:TLS packets that show the passage of Application Data (for example, in Netmon you can find application data packets via
.Protocol.TLS AND Description 'TLS:TLS Rec Layer-1 SSL Application Data' ). You should see a smooth progression in the time across the session. If you see long delays when refreshing your Outlook Online, this could be caused by a high degree of resets being sent.
Latency/Round Trip Time
Latency is a measure that can change a lot depending on many variables, such upgrading aging devices, adding a large number of users to a network, and the percentage of overall bandwidth consumed by other tasks on a network connection.
There are bandwidth calculators for Office 365 available from this Network planning and performance tuning for Office 365 page.
Office 365 Mac
Need to measure the speed of your connection, or your ISP connection's bandwidth? Try this site (or sites like it): Speedtest Official Site, or query your favorite search engine for the phrase speed test.
Tools
What to look for
To track latency in a trace, you will benefit from having recorded the client computer IP address and the IP address of the DNS server in Office 365. This is for the purpose of easier trace filtering. If you connect through a proxy, you will need your client computer IP address, the proxy/egress IP address, and the Office 365 DNS IP address, to make the work easier.
A ping request sent to outlook.office365.com will tell you the name of the datacenter receiving the request, even if ping may not be able to connect to send the trademark consecutive ICMP packets. If you use PsPing (a free tool for download), and specific the port (443) and perhaps to use IPv4 (-4) you will get an average round-trip-time for packets sent. This will work this for other URLs in the Office 365 services, like
psping -4 yourSite.sharepoint.com:443 . In fact, you can specify a number of pings to get a larger sample for your average, try something like psping -4 -n 20 yourSite-my.sharepoint.com:443 .
Note
PsPing doesn't send ICMP packets. It pings with TCP packets over a specific port, so you can use any one you know to be open. In Office 365, which uses SSL/TLS, try attaching port :443 to your PsPing.
If you loaded the slow performing Office 365 page while doing a network trace, you should filter a Netmon or Wireshark trace for
DNS . This is one of the IPs we're looking for.
Here are the steps to take to filter your Netmon to get the IP address (and take a look at DNS Latency). This example uses outlook.office365.com, but may also use the URL of a SharePoint Online tenant (hithere.sharepoint.com for example).
If you're going to Office 365 through a proxy server, the steps are a little different. You would first PsPing to your proxy server to get an average latency value in milliseconds to proxy/egress and back, and then either run PsPing on the proxy, or on a computer with a direct Internet connection to get the missing value (the one to Office 365 and back).
If you choose to run PsPing from the proxy, you'll have two millisecond values: Client computer to proxy server or egress point, and proxy server to Office 365. And you're done! Well, recording values, anyway.
If you run PsPing on another client computer that has a direct connection to the Internet, that is, without a proxy, you will have two millisecond values: Client computer to proxy server or egress point, and client computer to Office 365. In this case, subtract the value of client computer to proxy server or egress point from the value of client computer to Office 365, and you will have the RTT numbers from your client computer to the proxy server or egress point, and from proxy server or egress point to Office 365.
However, if you can find a client computer in the impacted location that is directly connected, or bypasses the proxy, you may choose to see if the issue reproduces there to begin with, and test using it thereafter.
Latency, as seen in a Netmon trace, those extra milliseconds can add up, if there are enough of them in any given session.
Note
Your IP address may be different than the IPs shown here, for example, your ping may return something more like 157.56.0.0/16 or a similar range. For a list of ranges used by Office 365, check out Office 365 URLs and IP address ranges.
Remember to expand all the nodes (there's a button at the top for this) if you want to search for, for example, 132.245.
Proxy Authentication
This only applies to you if you're going through a proxy server. If not, you can skip these steps. When working properly, proxy authentication should take place in milliseconds, consistently. You shouldn't see intermittent bad performance during peak usage periods (for example).
If Proxy authentication is on, each time you make a new TCP connection to Office 365 to get information, you need to pass through an authentication process behind the scenes. So, for example, when switching from Calendar to Mail in Outlook Online, you will authenticate. And in SharePoint Online, if a page displays media or data from multiple sites or locations, you will authenticate for each different TCP connection that is needed in order to render the data.
In Outlook Online, you may experience slow load times whenever you switch between Calendar and your mailbox, or slow page loads in SharePoint Online. However, there are other symptoms not listed here.
Proxy authentication is a setting on your egress proxy server. If it is causing a performance issue with Office 365, you must consult your networking team.
Tools
What to look for
Proxy authentication takes place whenever a new TCP session must be spun up, commonly to request files or info from the server, or to supply info. For example, you may see proxy authentication around HTTP GET or HTTP POST requests. If you want to see the frames where you are authenticating requests in your trace, add the 'NTLMSSP Summary' column to Netmon and filter for
.property.NTLMSSPSummary . To see how long the authentication is taking, add the Time Delta column.
To add a column to Netmon:
Even if you don't add the column, the Netmon filter will work. But your troubleshooting will be much easier if you can see what stage of authentication you're in.
When looking for instances of Proxy Authentication, be sure to study all frames where there is an NTLM Challenge, or an Authenticate Message is present. If necessary, right-click the specific piece of traffic and Find Conversations > TCP. Be aware of the Time Delta values in these Conversations.
A four second delay in proxy authentication as seen in Wireshark. The Time delta from previous displayed frame column was made via right-clicking the field of the same name in the frame details and selecting Add as Column.
DNS Performance
Name resolution works best and most quickly when it takes place as close to the client's country as possible.
If DNS name resolution is taking place overseas, it can add seconds to page loads. Ideally, name resolution happens in under 100ms. If not, you should do further investigation.
Tip
Not sure how Client Connectivity works in Office 365? Take a look at the Client Connectivity Reference document here.
Tools
What to look for
Analyzing DNS performance is typically another job for a network trace. However, PsPing is also helpful in ruling in, or out, a possible cause.
DNS traffic is based on TCP and UDP requests and responses are clearly marked with an ID that will help to match a specific request with its specific response. You'll see DNS traffic when, for example, SharePoint Online uses a network name or URL on a web page. As a rule of thumb, most of this traffic, except when transferring Zones, runs over UDP.
In both Netmon and Wireshark, the most basic filter that will let you look at DNS traffic is simply
dns . Be sure to use lower case when specifying the filter. Remember to flush your DNS resolver cache before you begin to reproduce the issue on your client computer. For example, if you have a slow SharePoint Online page load for the Home page, you should close all browsers, open a new browser, start tracing, flush your DNS resolver cache, and browse to your SharePoint Online site. Once the entire page resolves, you should stop and save the trace.
You want to look at the time offset here. And it may be helpful to add the Time Delta column to Netmon which you can do by completing these steps:
If you find a query of interest, consider isolating it by right-clicking that query in the frame details panel, choosing Find Conversations > DNS. Notice that the Network Conversations panel jumps right to the specific conversation in its log of UDP traffic.
In Wireshark you can make a column for DNS time. Take your trace (or open a trace) in Wireshark and filter by
dns , or, more helpfully, dns.time . Click on any DNS query, and, in the panel showing details, expand the Domain Name System (response) details. You'll see a field for time (for example, [Time: 0.001111100 seconds] . Right-click this time and select Apply as Column. This will give you a Time column for quicker sorting of your trace. Click on the new column to sort by descending values to see which DNS call took the longest to resolve.
If you would like to do more investigation of the DNS resolution time, try a PsPing against the DNS port used by TCP (for example,
psping <IP address of DNS server>:53 ) . Do you still see a performance issue? If you do, then the problem is more likely to be a broader network issue than an issue of specific the DNS application you're hitting to do resolution. It's also worth mentioning, again, that a ping to outlook.office365.com will tell you where DNS name resolution for Outlook Online is taking place (for example, outlook-namnorthwest.office365.com).
If the issue looks to be DNS specific, it may be necessary to contact your IT department to look at DNS configurations and DNS Forwarders to further investigate this issue.
Proxy Scalability
Services like Outlook Online in Office 365 grant clients multiple long-term connections. Therefore, each user may use more connections that require a longer life.
Office 365 Download For MacTools
Math
What to look for
There is no network trace or troubleshooting tool specific to this. Instead, it's based upon bandwidth calculations given limitations and other variables.
TCP Max Segment Size
Found in the SYN - SYN/ACK. Do this check in any performance network trace you've taken to ensure that TCP packets are configured to carry the maximum amount of data possible.
The goal is to see a MSS of 1460 bytes for transmission of data. If you're behind a proxy, or you are using a NAT, remember to run this test from client to proxy/egress/NAT, and from proxy/egress/NAT to Office 365 for best results! These are different TCP sessions.
Tools
Netmon
What to look for
TCP Max Segment Size (MSS) is another parameter of the three-way handshake in your network trace, that means you'll find the data you need in the SYN - SYN/ACK packet. MSS is actually pretty simple to see.
Open any performance network trace you have and find the connection you're curious about, or that demonstrates the performance problem.
Note
If you are looking at a trace and need to find the traffic relevant to your conversation, filter by the IP of the Client, or the IP of the proxy server or egress point, or both. Going directly, you will need to ping the URL that you're testing for the IP address of Office 365 in the trace, and filter by it.
Looking at the trace second-hand? Try using filters to orient yourself. In Netmon, run a search based on the URL, such as
Containsbin(framedata, ascii, 'sphybridExample') , take note of the frame number.
In Wireshark use something like
frame contains 'sphybridExample' . If you notice that you've found Remote Winsock (RWS) traffic (it may appear as a [PSH, ACK] in Wireshark), remember that RWS connects can be seen shortly before relevant SYN - SYN/ACKs, as discussed earlier.
At this point, you can record the frame number, drop the filter, click All Traffic in the Network Conversations window in Netmon to look at the nearest SYN.
Office 365 Slow Download Mac Installer
Importantly, if you didn't receive any of the IP address information at the time of the trace, finding your URL in the trace (part of
sphybridExample-my.sharepoint.com , for example), will give you IP addresses to filter by.
Locate the connection in the trace that you're interested in seeing. You may do this by either scanning the trace, by filtering by IP addresses, or by selecting specific Conversation IDs using the Network Conversations window in Netmon. Once you've found the SYN packet, expand TCP (in Netmon), or Transmission Control Protocol (in Wireshark) in the Frame Details panel. Expand TCP Options and MaxSegmentSize. Locate the related SYN-ACK frame and Expand TCP Options and MaxSegmentSize. The smaller of the two values will be your Maximum Segment Size. In this picture, I make use of the built-in Column in Netmon called TCP Troubleshoot.
The built-in column is at the top of the Frame Details panel. (To switch back to your normal view, click Columns again, and then choose Time Zone.)
Here's a filtered trace in Wireshark. There is a filter specific to the MSS value (
tcp.options.mss ). The frames of a SYN, SYN/ACK, ACK handshake are linked at the bottom of the Wireshark equivalent to Frame Details (so frame 47 ACK, links to 46 SYN/ACK, links to 43 SYN) to make this kind of work easier.
If you need to check Selective Acknowledgment (next topic in this matrix), don't close your trace!
Selective Acknowledgment
Found in the SYN - SYN/ACK. Must be reported as Permitted in both SYN and SYN/ACK. Selective Acknowledgment (SACK) allows for smoother retransmission of data when a packet or packets go missing. Devices can disable this feature, which can lead to performance problems.
If you're behind a proxy, or you are using a NAT, remember to run this test from client to proxy/egress/NAT, and from proxy/egress/NAT to Office 365 for best results! These are different TCP sessions.
Tools
Netmon
What to look for
Selective Acknowledgment (SACK) is another parameter in the SYN-SYN/ACK handshake. You can filter your trace for SYN - SYN/ACK many ways.
Locate the connection in the trace that you're interested in seeing either by scanning the trace, filtering by IP addresses, or by clicking a Conversation ID using the Network Conversations window in Netmon. Once you've found the SYN packet, expand TCP in Netmon, or Transmission Control Protocol in Wireshark in the Frame Details section. Expand TCP Options and then SACK. Locate the related SYN-ACK frame and Expand TCP Options and its SACK field. Make certain SACK is permitted in both SYN and SYN/ACK. Here are SACK values as seen in both Netmon and Wireshark.
DNS Geolocation
Where in the world Office 365 tries to resolve your DNS call effects your connection speed.
In Outlook Online, after the first DNS lookup is completed, the location of that DNS will be used to connect to your nearest datacenter. You will be connected to an Outlook Online CAS server, which will use the backbone network to connect to the datacenter (dC) where your data is stored. This is faster.
When accessing SharePoint Online, a user traveling abroad will be directed to their active datacenter -- that's the dC whose location is based on their SPO tenant's home-base (so, a dC in the USA if the user if USA-based).
Lync online has active nodes in more than one dC at a time. When requests are sent for Lync online instances, Microsoft's DNS will determine where in the world the request came from, and return IP addresses from the nearest regional dC where Lync online is active.
Tip
Need to know more about how clients connect to Office 365? Take a look at the Client Connectivity reference article (and its helpful graphics).
Tools
What to look for
Requests for name resolution from the client's DNS servers to Microsoft's DNS servers should in most cases result in Microsoft DNS returning the IP address of a regional datacenter (dC). What does this mean for you? If your headquarters are in Bangalore, India, but you are traveling in the United States, when your browser makes a request for Outlook Online, Microsoft's DNS servers should hand you IP addresses to datacenters in the United States -- a regional datacenter. If mail is needed from Outlook, that data will travel across Microsoft's quick backbone network between the datacenters.
DNS works fastest when name resolution is done as close to the user location as possible. If you're in Europe, you want to go to a Microsoft DNS in Europe, and (ideally) deal with a datacenter in Europe. Performance from a client in Europe going to DNS and a datacenter in America will be slower.
Office 365 For Mac Version
Run the Ping tool against outlook.office365.com to determine where in the world your DNS request is being routed. If you are in Europe, you should see a reply from something like outlook-emeawest.office365.com. In the Americas, expect something like outlook-namnorthwest.office365.com.
Open the command prompt on the client computer (via Start > Run > cmd or Windows key > type cmd). Type ping outlook.office365.com and press ENTER. Remember, to specify -4 if you want to specify to ping via IPv4. You may fail to get a reply from the ICMP packets, but you should see the name of the DNS to which the request was routed. If you want to see the latency numbers for this connection try PsPing to the IP address of the server that is returned by ping.
Office 365 Application TroubleshootingTools
We don't cover tools used in application-specific troubleshooting in this network-specific article. But you'll find resources you can use on this page.
Related TopicsTopics Map > Administrators Topics Map > Features and Functionality > Client Capabilities > Desktop Topics Map > Features and Functionality > Client Capabilities > Mobile Topics Map > Features and Functionality > Client Capabilities > Web (OWA)
Heavy calendar/email users have noticed performance issues when using Outlook and other desktop clients. Below are some steps that may improve performance in Outlook desktop clients when configured with an Office 365 (Exchange Online) account.
Important: Some of the information below asks you to make changes to your software and/or usage of them. We recommend that you discuss these changes with your local IT administrator prior to making these changes.
Other Resources
See Also:Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
December 2020
Categories |