Wednesday, February 17, 2016

DO NOT use Project Pro 2016 when syncing to SharePoint Online just yet

Project Pro for Office 365 2016 was released ~2 months ago.

As it came out slightly later than the whole Office 2016 suite (which had quite some troubles with OneNote syncing and OneDrive for Business), you would assume that it's a healthy product :)

Wrong.

In a synchronization with SharePoint Online, this thing simply won't work in your production.
If you sync a task or two, works fine. If you sync more than 50 (confirmed by MS) than it breaks the whole thing - your resources columns don't sync, same with the predecessors and that's just enough to stop you... there are other bugs with recurring tasks crashing the whole software, dates being messed up and so on.

I've raised this to MS a month ago and there's still no fix, they promised there will be a fix coming in one of the next public releases. To visualize what I mean..

All the resources are created in both the resource sheet in Project Pro and in the People Directory in SharePoint.

Project Plan in Project Pro 2016:


After the sync in SharePoint Online:


I'm not even going to start on the dates / times / completion stuff that breaks.

Now, after a downgrade to Project Pro 2013 - all works fine, however the installation of it is quite painful - you need to contact MS to get a link as it's not really available in the Office 365 portal as of today (unless you have applied for First Release on your whole tenant).

Tuesday, February 2, 2016

Migrating to Exchange Online (Part 1)

Yep, you got that right.

I volunteered and got involved  in a migration to Exchange Online these days and I thought it would be quick, nice and easy job to help the IT team in a mid-sized organization. Not even close!

The scope is the following:

3 IMAP e-mail solutions (5 domains, 3 geo-locations of the servers) -> Exchange Online

No migration tools and no external consultants will be involved in the migration.

The first thing you'd need to know when you plan and estimate your migration is speed. Depending on how important this is for you, you can choose the different options ahead.

Option 1: PST Import service 

Caveat: If your e-mail servers can't export a PST, then you need a 3rd party tool to do that.

Limitation: You can't import PST files larger than 10 GB or containing more than 1 000 000 items. So you might as well need a tool to split your largest mailboxes into multiple PST files.

1.1. Copy your PSTs to a secure Azure BLOB storage and then import them through the Admin Panel (you need to be a Global Admin)

We were considering to use the new Office 365 Import service launched last year (and still free, MS are planning to charge for it if you ship your drives) and we were very frustrated to find out the capping of 0.5 GB / hour when you import your PSTs. There's an additional prerequisite step to that, first you'd need to copy your PSTs in an Azure Blob, which for us worked as a charm - over 300 Mbps - and that's when the speed traffic police comes in and your happy face is gone :)

By speed traffic police I meant the Office 365 User Throttling - I still dunno how the PST import of 500 mailboxes, for instance could be considered user import. It's purely the migration team's job to export all PSTs, prepare PST mapping files and all that. I totally don't see it the same as if you'd import your data through Outlook - this is where there should the throttling apply, and not if these actions are associated from the Office 365 interface, but anyway... you do get the perks like calendars, tasks, contacts. It's never perfect.

Tip: If you run multiple PST import jobs at the same time - you get 0.5 GB / h for each of them.

1,2. Ship your hard drives to the nearest Microsoft data center and then import through the Admin Panel (still, you need to be a Global Admin)

This is usually the preferred method if you have slow bandwidth to Azure. On our tests, 300 Mbps were more than satisfactory and we don't use an ExpressRoute to Azure, This means that we can transfer 1 TB for ~13 hours. So in our particular scenario, shipping the hard drives would definitely take more than the copying to Azure, but if you have 10 TB of e-mail and slower connection to Azure, your choices are different.

If you still decide to go that dirt road (PST import in general), the official MS Import guide is here.

Option 2: IMAP migration

Obviously the only other option (I'm excluding user-handled migration through Outlook and all that, of course) is the IMAP migration. It comes in two flavors.

2.1. DIY e.g. get one of the many free 3rd party scripts out there and run it on your own environment - why would you do that?

This will get the source and target mailboxes in sync, but a big trouble with that is you need your source e-mail account passwords, and the destination (Office 365) passwords for the users as well. Not a big deal, unless your users are already in Office 365 and using stuff like SharePoint, OneDrive, Yammer, etc. A no-brainer to reset or ask them for their passwords.

2.2. Initiate the IMAP migration from the Exchange Admin Center

Before you start, a good survival guide about it is found here. The most obvious limitations:

  • Only items in a user's inbox or other mail folders can be migrated. You can’t migrate contacts, calendar items, or tasks.
  • A maximum of 500,000 items can be migrated from a user’s mailbox.
  • The maximum message size that can be migrated is 35 MB.
Provided this works for you, let's imagine you've already tried the PST import and you want to clear all stuff that you've imported and start anew. The most intuitive thing will be to delete the users - but they will be retained for about 30 days. A workaround is to rename and delete them :) 

Or... as a best practice empty those mailboxes through PowerShell.

Use the Search-Mailbox cmdlet and pipe it with DeleteContent.

Example: Search-Mailbox -Identity "John Doe" -DeleteContent

It's a bit slow, but maybe faster than opening the user's mailbox through Outlook (and for that you'd need to grant yourself Full Access to that mailbox first - lame and unprofessional approach).

So... the trouble with IMAP migration. If you run a single mailbox - the speed is still not much better - between 0.5 and 1 GB per hour. A disaster I'd say.

The key is concurrency - you'd need to run at least 20 mailboxes per batch so you can get your migration done in some realistic schedules that you can present to the business.

With the concurrency, we've easily reached speed of 10 GB per hour with 15 concurrent mailboxes. This totally corresponds with the Microsoft provided data, so you can bet on it. Of course, this excludes a lot of other factors which might block you from reaching such speeds - your ISP, your firewall, your current e-mail server bandwidth and so on and so forth.

In the process of setting this up we've found a nice little calc that helps you do the math.

So once you have your estimations and you prepare your CSV file for the IMAP migration, you start your sync. Everything goes well over the weekend, the Spring flowers are blooming and etc. NO!

There are many reasons why the IMAP sync can fail - wrong CSV format, wrong passwords, your e-mail server goes down, Microsoft Exchange Online goes down...but all of those you can either fix or just hope for Microsoft to fix as quickly as possible. But what to do when the sync fails and you want to start afresh?

Well, first the sync will not always stop if some of the above problems occur. It will continue running even if it runs fine for 1 mailbox and all the rest fail. So you need to stop it manually. From the GUI - I've noticed that stopping it manually leaves it in "Removing" state for more than 12 hours - dunno if it's a bug on our tenant or not, but then your only way is to kill it with PowerShell. The Remove-MigrationBatch command comes handy in this case. If you miss this part, your new sync with all the correct data will NOT run as the same mailboxes are already part of another sync which is not totally gone. Use the "-force" parameter, otherwise there is no result (at least in our tests).

Once you clear the migration job, you go to the GUI in EAC, you see it gone and you import your same CSV file and hope to get this running. No! 2 reasons:

If you import the same CSV file, EAC tries to be smart and doesn't detect the mailboxes which were part of the previous synchronizations. So rename your CSV file.

Your next attempt would also fail. Even if your mailboxes get detected and they go into the batch, they will fail. Don't ignore the error report, but download it and read the following:

The user "dimitar.miriyski@abc.xyz" already exists, but the migration batch that includes it couldn't be found.

Does that make sense? Yes, of course. Your Office 365 user is not deleted. But the migration batch is. So what's the problem with that? There's a history of all those batches... which you need to sweep off manually with the Remove-MigrationUser command. Hopefully you haven't messed with too many users in your failed batch :) Ideally it should never have failed, but things like that happen.

Another trick when running the migration batches is not to use CAPS in the batch title.

Option 3: A mix of 1 and 2.

It seems as the perfect cocktail - you do the bulk stuff with IMAP sync to gain the speed advantage and then you do the calendars, tasks and contacts through PST export and import. Not tested yet.

Conclusion

So... from the short time of research and testing this, I'd say there are a lot of things you'd need to consider and Exchange Online is not a toy. No wonder the Exchange Administrators are still out there, but in the next post from this series I'll see if I can prove that a SharePoint guy with some Office 365 knowledge and experience can do the trick for a small to medium sized company.

Tuesday, January 26, 2016

Syncing fields from Project Pro to SharePoint Online

Something that started as a quite simple requirement turned out into a more hefty task, so I decided to share it here.

So the requirement: Be able to sync fields from Project Pro to a SharePoint Task List in SharePoint Online. All great, works out of the box, with the exception of 2 things:

- If the column in SharePoint is of type "Multiple lines of text" and the type of text to allow is "Enhanced rich text (Rich text with pictures, tables, and hyperlinks)" that doesn't work correctly.

- If the column is of type "Person/Group" then this doesn't work as expected. Some resources are missing on consecutive synchronizations from Project Pro to SharePoint Online. I've taken this with MS to see what's going wrong, but for now if it happens to you - configure your resource or "Assigned To" (named by default) column to be of type "Single line of text" or think of it as just a string.


You will loose the functionality to check the person's status in Skype for Business (if you use that) and to click on the user profile and go to their Delve page (in Office 365). But you'll still be able to sort and filter on that column if you want to view a particular person's tasks.


I'll describe the steps needed to match the requirement to sync Task Descriptions to SharePoint Online. 


By default, the Description column in the Task Lists (coming with the default Task Content Type) is of type Multiple lines of text and has the "Enhanced rich text (Rich text with pictures, tables, and hyperlinks)" as the allowed type of text. I've just renamed the column in this example:







Fill in some tasks if you're the PM (or just one if you're a lazy blogger like myself):




Then open the Task List with Project Pro (provided you do have the license assigned and the software installed from the Office 365 Portal):





You won't see your description column:





When you go to File -> Info -> Map Fields, you won't be able to map anything to it:



Now, part of the solution is to change your column allowed text type to Plain Text in SharePoint.





But even then, you won't see it in Project Pro the next time you open your project plan.

The key is in the mappings. Add a new column in Project Pro. You can choose some of the predefined text columns, e.g. Text1:





Name it whatever you want, let's do Task Description to avoid any confusion. Hint: the columns in Project Pro and SharePoint Online don't really have to be using the same naming convention, as long as they're mapped. Right click on your newly created column and go to Field Settings (and no, double clicking on the column doesn't do what double-clicking on a row does, sorry):





Tile field - self explanatory:






Now, when you go back to the Map Fields you can definitely map your newly created column in Project Pro (although its name is not reflected in the mapping list - dunno why) to your SharePoint Online column which appears nicely as Task Description:





Once you're done with all your mappings, just click the huge Save button.

If you've tried to break things like me and inserted different values in the field in SharePoint Online and Project Pro, then you'll get this good warning. Skip this if you're a good user and didn't do that :)

The warning basically is quite liberal - it gives you the option to choose whether you like your Project Pro or your SharePoint Online value better and it's even displaying them and color-coding them for you. I'll keep the Project Pro value which says "Sync this description":





Boom! It's done.




(Optional) If you want to break things even further, you can go back to your column in the SharePoint Online Task List and change it to allow Enhanced Rich Text again and then open the Task List with Project Pro. You'll see this (and believe me, it gets even uglier when you include some links, etc.)





Hope that's helpful if you stumble upon it, I'll keep another post once I sort out the issue with syncing resources without using the string field workaround with MS (if at all).

Wednesday, January 6, 2016

New Year...OneDrive new UI and a bug in it

Update 01/02/2016: It seems that through the new UI you can now even upload folders (one by one for now) with drag & drop! Great functionality that has been missing so far...

Hey, it's been a while since my last post...holiday season got us all, right? :) Happy New Year, everyone! Let it bring you a lot of new tech, gadjets, innovations and of course health and happiness!

It's been more than a month since we all could use the OneDrive new UI and I've decided to share my first positive impressions of the it and a very unpleasant bug unfortunately, so you can take an informed decision whether to use it or switch back to the classic experience.

First, it looks really nice, based on HTML5 and it's even responsive! Good job so far for the OneDrive team. There must be something with Jeff Teper going back to leading the SharePoint and OneDrive divisions @ MS :)

The menus are arranged in a better way, even though the Version History is still kind of hidden in the context menu.












You can now drag & drop stuff in the browser... e.g. move a file or folder into another one - how cool is that? Let's hope we'll see that in SharePoint some day... which is still using some old-school XHTML 1.0.




There's also a file size column now added, which I don't use that much, considering the 1 TB storage limit, but anyway... another small improvement.

Now tot he bad news...

Whenever you try to share a folder (and it happens with files sometimes) and share it with more than 1 person they DON'T all get the notifications over email. E.g. you share it with Peter, Joe and Dean, then you check your mail (as you're in cc of those notifications by default) and realize that only Peter got the email with the link to the file.

Unpleasant.

After I've talked with one of my friends at Microsoft, I got a confirmation that this issue is only present in the new UI and is being worked upon, however it's now taking a few weeks and there's no resolution.

It all works just fine if you do share the file/folder with one person. And of course you can always forward the notification to the appropriate people once you see it in your inbox, but this is what OneDrive's for - avoid shooting a bunch of emails about the same file.

I'm hoping to see this resolved soon, otherwise we'll just have to use the classic experience...which is not that sophisticated to be honest, but at least one of its basic functionalities (the notification) works.

I might write another post about the NGCS (Next Generation Sync Client) that I was testing as part of the preview for more than a month now... still some mixed feelings about it, if you plan to use both the new UI and the new client organization-wide, maybe you should hold on...

Thursday, December 3, 2015

Office 365 down in Europe

Today Office 365 went down in Europe and it's down since more than 3 hours at the time of this post.

OK, Microsoft, it's technology - we get it, and you'd still make the numbers in the SLA report due to the high number of users you've got.

But there's something more worrying in today's scenario - communication!

Wouldn't it be nice to put up something on the http://status.office365.com page rather than the bogus message below? And not asking people to login to their tenants and check their dashboards? When in fact they CAN'T login at the time of calling you...


People are relying on this platform for their daily jobs, and a good communication would have been key in user satisfaction with the products. The communications today is absolutely ridiculous.

I even got a call from one of your support reps asking me to do a remote session and see one of our other issues we're experiencing, so I kindly had to inform him everything is DOWN in Europe!

Meanwhile, we had to find the hard way what's working and what's not, e.g;

- OneDrive sync is working
- OneNote sync is working
- Yammer mobile app is fine (if you haven't logged out)
- Everything else is FINE if you were logged in and haven't logged our prior to 11:00 this morning.

I'd say this is a complete communication failure on your end today...



And while you've put this nice little "warning" message on the Azure portal status, it's still unclear and doesn't mention Office 365 (you need to be aware it's dependent on Azure) and it's saying North and West Europe...while NO ONE in Europe can't go to the Login Portal.


Monday, November 23, 2015

1 TB space in SharePont Online! /not OneDrive/

Update 18/02. The space has reappeared (this time as 1 TB - expected) although the feature on the roadmap is still "In Development".




Update: Microsoft has reclaimed this space and when talking to them, they said this was "a test 2 TB rollout to select tenants", and ours is not a First Release one. Shame.

Anyway, they said by the end of January all tenants (first they claimed Enterprise only) should get upgraded to 1 TB.

On this foggy November morning, I've logged in to the admin panel in one SPO tenant and saw this!


Prior to Friday afternoon, that space was ~130 GB, based on the well-known formula 10 GB + 0.5 GB per licensed user. 

The O365 roadmap says Office 365 tenants will now get 1TB + (500MB * # of E/A/G users) but this feature is still in development, and our tenant is not a FR one.

Have you already observed the storage increases in your tenants?

By the way, thanks Microsoft! Really generous :)

Friday, November 6, 2015

People Directory with CSWP

Today I started to work on a mock-up of a People Directory which is not utilizing anything else, but a Content Search Web Part due to the easy customization Display Templates and the powerful features it offers. It already has som good templates like Large picture for example, that will be all I need + Department and Job Title (which will be managed properties, linked to a term set).

Even though this Web Part is heavily promoted and is supposed to enhance/replace the good old CQWP, it doesn't work right away as you'd expect (at least in SharePoint Online).

Let's imagine you create a blank page and you add the Web Part.

You then configure your query - it'll be very simple "@*onmicrosoft.com" so that I can get all the people from my demo tenant

As you can see, the query is tested and it simply works. Then you save your Web Part and get this:


Some people suggested that this Web Part is not available in SharePoint Online Plan 1 and they've been in touch with Microsoft Support .... which I have to use in production... but I couldn't find an official source stating that. I think that's old information, I think the web part was not even available in SharePoint Online in the beginning. I know it's definitely not available on-premise if you don't have the Enterprise version.

I've also tested this on my demo tenant which is on SharePoint Online Plan 2... and I got the same results. Turns out pretty simple, I just had to enable the Cross Site Publishing feature on the site collection and the web part works.

+ a little bit of CSS = a sample of my desired outcome:


The next thing is to find out which properties we want displayed in these boxes, if they're custom ones they should be mapped to a managed property first before we can stick them in here.


But that will be another blog post :)