onsdag 6 november 2013
onsdag 13 juni 2012
Business Intelligence in SharePoint
The BI offering in SharePoint is not related to a single specific technology or product. SharePoint provides a wide array of offerings that can help customers solve a variety of problems. SharePoint therefor presents us with questions that are hard to answer when it comes to BI. For instance: which of the seven ways of creating KPI’s in SharePoint should a customer use? This article will give a brief introduction to the different kinds of BI SharePoint can provide.
Bi overview
The purpose of BI is to provide the business with metrics that can be used for analysis and steering. KPI’s allow a business to measure pre-defined metrics, such a s sales for a region or the percentage of staff utilization at a given point in time. It’s a way of highlighting what is happening so that future corrective action can be undertaken by the business.
Business Intelligences role in the organization can be mapped to the steering process. Let’s exemplify this by thinking about how a business can answer these questions:
1. What has happened?
2. What is happening?
3. Why is this happening?
4. What will happen?
5. What do we want to happen?
Questions 1 and 2 are plain monitoring activities, business intelligence can support this process by providing an efficient and trustworthy visualization of data. To answer question 3 we need to perform an analysis, this is done by selecting a subset of data that can be used to explain the selected phenomenon. Now that we have established the past, present and the phenomenon driving this trend we can focus on what we want to happen in the future.
All these activities rely heavily on getting the correct data in the most efficient way possible. Normally data is stored in an unstructured way; for instance in accident-reports or ERP systems.
BI transforms this raw data into something that is meaningful for humans and allows managing of data. In the SharePoint context we can simplify this to:
BI provides the tools that enable individuals, teams and enterprises to discover and analyze information for:
- Decision making
- Analysis
- Collaboration
- Action
The case for SharePoint as a BI tool is that it enables us to create and manage information in an integrated system that includes core business productivity features such as collaboration, search and content management. What we really gain from SharePoint is a BI tool is insight into data.
Microsoft’s likes to stress that SharePoint empowers people by allowing them to access BI data through a familiar interface while working with the tools that they already know how to use, like Excel and Visio.
Users create BI assets in SharePoint with tools like PowerPivot, Excel, Visio, Report Builder and PerformancePoint Dashboard designer which are stored securely in SharePoint lists and libraries.
An example of how a BI solution can be leveraged is described below:
1. SCOM is used to collect state and performance counter data from managed server.
2. Collects data from the manages servers, pre-aggregates and stores in tables designed to support reporting requirements
3. Small subset is transformed and loaded into BI Framework database. Contains star schemas for the Analysis Services OLAP cubes
4. Cubes are built and processed
5. Data from cubes are used to populate PerformancePoint Server scorecards, dashboards and reports
6. Scorecards, dashboards and reports are made available to the users through SharePoint. Refreshed when new data is available in the cubes
To meet this scenario using SQL Server Reporting Services running as a separate solution or integrated in SharePoint is a good solution.
Information consumers tend to explore the data more than information users. But they do not have the required skills to query the database themselves.
To meet this scenario use interactive reports that have parameters that can be set by the user to filter and sort data. These reports also include a way to dig deeper into the data, either by displaying them in the same report or opening a separate report. SQL Server Reporting Services can facilitate this need. We can think if this user as perhaps a staff manager that only wants to tweak the data slightly to see how performance is related to his/her leadership role.
Power users need maximum interaction possibilities with the data. This includes building their own queries or even generating reports for other groups of users. To facilitate this Report Builder or Excel 2010 with PowerPivot should be used. PowerPivot is best suited for scenarios where multiple data-sources are used. Now this user has a higher need of seeing larger amounts of data from several different angles, often he/she is the only or one of few that needs this data to do their job, perhaps it is a controller from the finance department or a buyer that uses numbers to get the best deals? Now this user can create his/her own ad-hoc BI solutions, using for instance a large excel sheet as the datasource.
For users that needs highly summarized and processed data it’s recommended to use Reporting Services or dashboards/scorecards in SharePoint Server 2010 or PerformancePoint services.
The point is that when we think of BI we usually think of centrally configured solutions that are set up and used. But BI in SharePoint enables so much more, as long as we think about who the user is and what they are really trying to achieve. The market for these kinds of solutions is huge and growing every day. It feels nice that we are on board and starting to deliver value to our customers together with the BI team in this area.
Bi overview
The purpose of BI is to provide the business with metrics that can be used for analysis and steering. KPI’s allow a business to measure pre-defined metrics, such a s sales for a region or the percentage of staff utilization at a given point in time. It’s a way of highlighting what is happening so that future corrective action can be undertaken by the business.
Business Intelligences role in the organization can be mapped to the steering process. Let’s exemplify this by thinking about how a business can answer these questions:
1. What has happened?
2. What is happening?
3. Why is this happening?
4. What will happen?
5. What do we want to happen?
Questions 1 and 2 are plain monitoring activities, business intelligence can support this process by providing an efficient and trustworthy visualization of data. To answer question 3 we need to perform an analysis, this is done by selecting a subset of data that can be used to explain the selected phenomenon. Now that we have established the past, present and the phenomenon driving this trend we can focus on what we want to happen in the future.
All these activities rely heavily on getting the correct data in the most efficient way possible. Normally data is stored in an unstructured way; for instance in accident-reports or ERP systems.
BI transforms this raw data into something that is meaningful for humans and allows managing of data. In the SharePoint context we can simplify this to:
BI provides the tools that enable individuals, teams and enterprises to discover and analyze information for:
- Decision making
- Analysis
- Collaboration
- Action
The case for SharePoint as a BI tool is that it enables us to create and manage information in an integrated system that includes core business productivity features such as collaboration, search and content management. What we really gain from SharePoint is a BI tool is insight into data.
Microsoft’s likes to stress that SharePoint empowers people by allowing them to access BI data through a familiar interface while working with the tools that they already know how to use, like Excel and Visio.
SharePoint in the BI context
SharePoint can be used with SQL Server reporting and BI tools to surface BI data in ways that are meaningful. A back-end system such as SQL Server Reporting Services provides the data infrastructure that allows report authors and business users trusted, scalable and secure data.Users create BI assets in SharePoint with tools like PowerPivot, Excel, Visio, Report Builder and PerformancePoint Dashboard designer which are stored securely in SharePoint lists and libraries.
An example of how a BI solution can be leveraged is described below:
1. SCOM is used to collect state and performance counter data from managed server.
2. Collects data from the manages servers, pre-aggregates and stores in tables designed to support reporting requirements
3. Small subset is transformed and loaded into BI Framework database. Contains star schemas for the Analysis Services OLAP cubes
4. Cubes are built and processed
5. Data from cubes are used to populate PerformancePoint Server scorecards, dashboards and reports
6. Scorecards, dashboards and reports are made available to the users through SharePoint. Refreshed when new data is available in the cubes
Different users and self-service BI
The casual users of BI in SharePoint are information users, they don’t use BI daily and are mostly interested in having data presented to them that someone else have already set up and published in a central location, like a BI designer. Think of this user as the normal 9-5 employee that opens the intranet in the morning, gets a quick overview of the whole companies performance down to their own department.To meet this scenario using SQL Server Reporting Services running as a separate solution or integrated in SharePoint is a good solution.
Information consumers tend to explore the data more than information users. But they do not have the required skills to query the database themselves.
To meet this scenario use interactive reports that have parameters that can be set by the user to filter and sort data. These reports also include a way to dig deeper into the data, either by displaying them in the same report or opening a separate report. SQL Server Reporting Services can facilitate this need. We can think if this user as perhaps a staff manager that only wants to tweak the data slightly to see how performance is related to his/her leadership role.
Power users need maximum interaction possibilities with the data. This includes building their own queries or even generating reports for other groups of users. To facilitate this Report Builder or Excel 2010 with PowerPivot should be used. PowerPivot is best suited for scenarios where multiple data-sources are used. Now this user has a higher need of seeing larger amounts of data from several different angles, often he/she is the only or one of few that needs this data to do their job, perhaps it is a controller from the finance department or a buyer that uses numbers to get the best deals? Now this user can create his/her own ad-hoc BI solutions, using for instance a large excel sheet as the datasource.
For users that needs highly summarized and processed data it’s recommended to use Reporting Services or dashboards/scorecards in SharePoint Server 2010 or PerformancePoint services.
The point is that when we think of BI we usually think of centrally configured solutions that are set up and used. But BI in SharePoint enables so much more, as long as we think about who the user is and what they are really trying to achieve. The market for these kinds of solutions is huge and growing every day. It feels nice that we are on board and starting to deliver value to our customers together with the BI team in this area.
fredag 1 juni 2012
How To Prevent Visual Studio 2012 ALL CAPS Menus!
One of the most important blog posts related to the Visual Sutdio 2012 RC - How To Prevent Visual Studio 2012 ALL CAPS Menus!
onsdag 30 maj 2012
Why will the content type associated to the Pages library update?
This was a simple problem that a college had. When changes were made to the content type it did not push through to the "Pages" library.
First we create the new site column Content_HU
First we create the new site column Content_HU
<Field ID="{801797DA-B4A3-45F3-A177-4D405C85C2D9}"
Name="Content_HU"
DisplayName="$Resources:Company.Intranet.Resources,column_Content_HU;"
Type="Note"
Required="False"
Group="Company Intranet"/>
Add it to content type
<FieldRef ID="{801797DA-B4A3-45F3-A177-4D405C85C2D9}" Name="Content_HU" />
Added richtextfield (or PublishingWebControls:RichHtmlField if
we use the html type) for Content_HU in page layout
<SharePointWebControls:RichTextField ID="RichTextField1" FieldName="Content_HU" runat="server"/>
Now when we check the associated content type in Pages
library of site test (publishing site under site collection) we can see that
Parent Company.Intranet.ContentTypes.NewsPage3
English Content Multiple lines of text Optional
Hungarian Content Multiple lines of text Optional
Swedish Content Multiple lines of text Optional
Child Company.Intranet.ContentTypes.NewsPage3
English Content Multiple lines of text Optional Company.Intranet.ContentTypes.NewsPage
Swedish Content Multiple lines of text Optional Company.Intranet.ContentTypes.NewsPage
As we can see, the changes have not been pushed out to the
library. This is because the content type associated with the library is a copy
of the site content type, and since it made the copy before the new one existed
nothing is pushed through.
Now if I make another publishing site test2 under the site
collection it will inherit from the latest version of the content type
associated with the page layot. And thus letting me use it
If we really need to push updates like this we should use upgrade
actions.
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"> <UpgradeActions> <VersionRange EndVersion ="2.0.0.0"> <AddContentTypeField ContentTypeId="0x0101002b0e208ace0a4b7e83e706b19f32cab9" FieldId="{ccbcd479-94c9-4f3a-95c4-58897da434fe}" PushDown="True"/> </VersionRange> </UpgradeActions> </Feature>
This guy has a blogpost about it that pretty much shows it, otherwise the rest of the article the above snippet is from is findable on MSDN. But of course people are still having trouble with actually getting this to work, this is when we turn to questions on one of the stackexchange sites (like this one).
onsdag 9 maj 2012
A collection of nuggets
Nuggets
Nuggets are posts with links I stumble across and want to remember in the future, either for reference or because they might provide input for longer ramblings. Now I don't know if this is something blogworthy or not, if you stumble across this and consider it an idiotic idea, let me know in the comments.Deployment
Jan Steenbeek explains how to trick SharePoint into deploying solutions to target a specific web application instead of making it globally available in this article. Good tip. Combine it with these simple powershell deployment scripts from Softartisans and go deploy.Kill the mobile interface
SharePoint Joel has figured out that noone uses WAP anymore and urges us to "Kill SharePoint 2010 Mobile Interface". I couldn't agree more.Dependency injection
Dependency Injection and inversion of control is something that confused me for a long time. James Shore has an introductory article on the subject where he says: "Dependency injection means giving an object its instance variables. Really. That's it.". And in it's simplest form, this is actually true.At the end of the article he points to further reading that goes into depth on the subject. This is something that is widely used in the .NET community and something that I will come back to in a blog entry in the future.
Spsdatagenerator
I have been using the datagrid to mass input test data when testing out solutions. Now I don't have to do this anymore thanks to the spsdatagenerator project on codeplex. Hurrah!lördag 28 april 2012
SharePoint, TDD, code monkeys and Agile
I have been reading Eric Shupps (albeit old) thoughts on the matter of SharePoint and TDD as well as Andrew Woodwards series of articles about the subject. Thought I would share a few words on the subject since I find it truly fascinating.
First up though. There is a difference between unit testing and TDD. I don't subscribe do a single school or train of thought about TDD. In my head TDD is a way to use unit testing to achieve greater results then by writing tests to validate code already written. TDD can drive design, test should be written and fail before code is written. Unless you are doing exploratory coding (ie when you are not sure what the test case is. It's a good point to try and think more about what you are doing before diving into the code since this can lead you down a pretty deep rabbit hole).
While Eric gives a good summary of what he means with TDD and where it stems from I have a few issues that popped into my mind while reading his article that I want to study in more depth.
In the blog post mentioned above he quickly concludes the following regarding pair programming:
"First of all, companies quickly discovered that pair-programming is a tremendously inefficient process." [...] "putting two of them [developers] on the same coding task is rarely cost-effective".
And then he ends with:
"TDD assumes that everyone in the development lifecycle both understands and is willing to assume the overhead of enforcing the restrictions of the methodology"
First up though. There is a difference between unit testing and TDD. I don't subscribe do a single school or train of thought about TDD. In my head TDD is a way to use unit testing to achieve greater results then by writing tests to validate code already written. TDD can drive design, test should be written and fail before code is written. Unless you are doing exploratory coding (ie when you are not sure what the test case is. It's a good point to try and think more about what you are doing before diving into the code since this can lead you down a pretty deep rabbit hole).
While Eric gives a good summary of what he means with TDD and where it stems from I have a few issues that popped into my mind while reading his article that I want to study in more depth.
Pair programming
First a few notes about pair programming (which doesn't really have a 1:1 mapping to TDD but is still an interesting topic).In the blog post mentioned above he quickly concludes the following regarding pair programming:
"First of all, companies quickly discovered that pair-programming is a tremendously inefficient process." [...] "putting two of them [developers] on the same coding task is rarely cost-effective".
And then he ends with:
"Finally, it has been my personal experience that two coders working side-by-side tend to engage in more discussion about how things should be done in a perfect world than they do actually writing any code. They are by their nature perfectionists, and each always wants to prove that they have the best and most innovative ideas."
I was a bit sad to not see any references to these claims beyond his own experiences, in general I find that it's quite hard to find any sources at all on the subject.
I was a bit sad to not see any references to these claims beyond his own experiences, in general I find that it's quite hard to find any sources at all on the subject.
The way that the Eric poses the problem it is more a problem of communication and ego, not the pair programming paradigm in my humble opinion. Angela Harms has done a really great talk about this subject at the Ruby Midwest conference as well as an appearance on the Ruby Rogues podcast.
If I attempt to summarize her words bluntly she says that for pair programming to function you need to check your ego at the door. The process is not about finding the perfect solution, it's about figuring out how to move forward and combining brainpower. Being able to admit that you don't understand what is going on and that you don't know everything is step one in this process. The real point of pair programming and Agile in general is to bring problems to the surface early. If noone is admitting that the problems exists, you can't do anything about it.
Another great resource that should is the talk from Agile Roots named What We Learned in 8829 Hours of Pair Programming.
This is true and also the main point of Eric's post. A lot of my writing below could be considered nitpicking, but I like to see it more as a way to show what I object to.
"You can see this in action by listening to any TDD adherent as they bandy about phrases like "100% code coverage", "atomic validation", "no debugging" and other such phrases designed to reinforce their own misguided sense of infallibility."
In my humble opinion code coverage is not and should not be an end goal. 100 % code coverage can even be a bad thing. It's interesting thought, the whole time I was reading Eric's article I couldn't stop thinking about how different our views on TDD and it's evangelists is. Perhaps this has something to do with that I got into TDD from the ruby/rails community where TDD/BDD is the norm and the debate is around the concept of TDD and how it should be applied. In my mind there is no one true way of doing TDD.
Another great resource that should is the talk from Agile Roots named What We Learned in 8829 Hours of Pair Programming.
TDD Zealotry
"Developers who become adherents to the new programming religion of the moment quickly slip into the fatal trap of assuming, despite all evidence to the contrary, that their methodology makes the code they create bulletproof"This is true and also the main point of Eric's post. A lot of my writing below could be considered nitpicking, but I like to see it more as a way to show what I object to.
"You can see this in action by listening to any TDD adherent as they bandy about phrases like "100% code coverage", "atomic validation", "no debugging" and other such phrases designed to reinforce their own misguided sense of infallibility."
In my humble opinion code coverage is not and should not be an end goal. 100 % code coverage can even be a bad thing. It's interesting thought, the whole time I was reading Eric's article I couldn't stop thinking about how different our views on TDD and it's evangelists is. Perhaps this has something to do with that I got into TDD from the ruby/rails community where TDD/BDD is the norm and the debate is around the concept of TDD and how it should be applied. In my mind there is no one true way of doing TDD.
It's perhaps fitting that the greatest battle is fought inside the ranks of the same militia? (Follow the link-trail around DHH's "Testing like the TSA" article to see what I mean).
"It is vitally important to emphasize that unit testing is only one part of the overall quality assurance lifecycle."
This actually neatly summarizes the whole argument.
This actually neatly summarizes the whole argument.
Quality and the process of software development
Eric then goes on to reason about unit tests in relation to functional requirements and that writing the minimum amount of code is what give us unpredictable bugs among other things that I found hard to see the reasoning behind. But then he states that:
"I'm sorry to point out this blatantly obvious fact but developers don't really know squat about quality and even less about how to achieve it.".
This made me very upset, not at Eric but at the state of the developer in the world as seen from the eyes of some non-developers. The notion of developers as code monkeys that only care about solving the immediate problem at hand, system or users be damned always upsets me. If your developers have this approach to writing software, everything Eric is talking about up to this point is true and makes sense (although it's not related to the TDD practice).
Granted, there are lots of developers in the world who sees it as a job and nothing more. But why shouldn't developers care about quality? Do we not want developers who care about quality? The customers problem we are solving? The value we are bringing to the customers business? Why would we not want to promote something that makes developers think about these questions?
Granted, there are lots of developers in the world who sees it as a job and nothing more. But why shouldn't developers care about quality? Do we not want developers who care about quality? The customers problem we are solving? The value we are bringing to the customers business? Why would we not want to promote something that makes developers think about these questions?
Interestingly enough this is the point he is actually arguing in the article. The way we are working is broken, the way we conduct software engineering is broken. But somehow he manages to wrestle this into an argument that unit testing only proves that an individual part work but not that it will work as a part of a holistic system. True, this is not what unit tests are about, this is functional and integration testing.
TDD, more than code and unit tests
The thing about TDD is this, it provides more value then just testing the individual parts. The tests that come out of the process are a great compliment to documentation. Especially if the TDD practice of red, green, refactor has been implemented.
Imagine having a test suite with passing tests, when you change something you can see if the unit still works as intended, even after the code has been changed. As long as we have passing tests we have documentation that actually represents the current snapshot of the system. This is what allows TDD to drive design, when we decouple and abstract things we can be confident that the unit still behaves as intended. This only holds true if the unit tests actually test our logic, not just snippets here and there.
Tests are a great supplement to documentation, if the documentation is out of date we can check the tests to see how it differs, or if we read the tests first the documentation will make more sense since we can see what approach the developer who wrote the code took to the problem.
Real testers are great for really bringing the wrath of god (synonym to putting the system in the hands of a user) upon our system. But what do testers really know about the details of the code? Why should they care? Other developers care about what other developers have done.
Understanding how the system works as a whole is crucial to the successful launch of a system. Although I don't see anyone arguing otherwise.
It's also worth noting that while TDD does drive a certain style of design (that can go overboard just as Eric talks about) it's always up to developers/architects to be pragmatic and ensure that the design still has a point and brings value.
Agile, not just a word
Eric do strike a good point that I would like to expand somewhat on (this is where I go on a side-track):
The key words in this quote is understands and is willing. Can you see where I am going with this?
It is my firm belief that Agile in general wont work when people don't care or understand. When you don't care you are not going to put in more effort then you have to to keep your job. When you don't understand you rely on someone guiding you and telling you exactly what to do.
Of course pair programming, TDD or Agile in wont work when you don't have the people you need. Agile does not work in an 18:th century coal mine or the corporation where you have to sign a sheet of paper to go the bathroom.
I also object to the general thinking that we should only meet the required functional requirements. This kind of thinking assumes that the software development life-cycle is based around:
Order it from company X -> Create it -> Launch it -> Use it -> Throw it away -> Repeat with company Y, return to X when everyone has forgotten what they did last time.
The process should be:
Need it -> Think about it -> Build the minimum that you need -> Use it -> Think about it -> Build the next minimum you need -> Think about it
Testing SharePoint
Back to the point at hand.
Here I present a few simple rules for using TDD or even straight up unit testing within a SharePoint project:
- Don't over-test the SharePoint API, test your logic
Don't test the retrieval of SPListItems from the Contracts list. Extract into ContractItem's and test those. Focus on core logic and have faith that Microsoft has tested it's internal API's. Mock where needed, not everywhere. - Don't measure code coverage.
Code coverage is arbitrary and says nothing about what we are testing or why we are testing it. - At least test the happy path.
If I give this method this input, does it work? - Write enough tests so that you feel comfortable with the code you wrote.
We do after all think about what is going to happen when we write code, let's put the assumptions into tests. I dare anyone to write a non trivial regexp and know if it works without testing it. - Remember the refactor in Red, green, refactor.
The first passing test is not a signal to move on to the next piece of code. It's the signal to refactor the code that you just wrote.
The list is quite far from what a purist TDD evangelist would describe. But I am pragmatic, and I think that until better tools are provided that allows us to do true TDD, we should stay in this kind of grey area of TDD.
I also agree with Eric about the fact that TDD isn't a silver bullet to anything, especially not quality. To write good code you have to know how to write good code. You learn how to write good code by writing code and learning from your seniors. This however raises the point that using TDD by itself is pointless unless you take other Agile practices to heart. TDD must be combined with things like code reviews, pair programming and the philosophy of continuous improvement to create good coders.
Talking about TDD without talking about the software philosophy surrounding it is like talking about how to reduce the price of wood without knowing what your building.
Now go read part 2 where Eric clearly has had time to reflect and formulate his points in greater detail.
onsdag 25 april 2012
Agave web extensions
The Microsoft enterprise blog has an article up about Agave web extensions and Office programmability that I took a shine to.
The integration between the Office suite and SharePoint have been missing an easy way for programmers to create solutions that brings the two together. These new shiny rumored objects are interesting.
Lets consider them from the context of the relationship that currently exists between Excel and the SharePoint BI offering. Microsoft could have tried to move people away from Excel to more powerful tools better suited for the purpose of creating business intelligence solutions. But instead they choose to allow people to leverage Excel and PowerPivot as powerful BI self-service solutions where a simple Excel sheet can act as a full fledged datasource. The philosophy of letting people use the tools they are familiar with to provide business value as well as offering more advanced tools that can be used when needed seems to be the underlining strategy going forward.
The new programmability could bring the same mindset into the rest of Microsoft's offering. Extend, don't reinvent.
Further tying SharePoint into the stack as the center of information distribution and collaboration seems like a great strategy, not just for us active in the business, but for the enterprise as well.
The integration between the Office suite and SharePoint have been missing an easy way for programmers to create solutions that brings the two together. These new shiny rumored objects are interesting.
Lets consider them from the context of the relationship that currently exists between Excel and the SharePoint BI offering. Microsoft could have tried to move people away from Excel to more powerful tools better suited for the purpose of creating business intelligence solutions. But instead they choose to allow people to leverage Excel and PowerPivot as powerful BI self-service solutions where a simple Excel sheet can act as a full fledged datasource. The philosophy of letting people use the tools they are familiar with to provide business value as well as offering more advanced tools that can be used when needed seems to be the underlining strategy going forward.
The new programmability could bring the same mindset into the rest of Microsoft's offering. Extend, don't reinvent.
Further tying SharePoint into the stack as the center of information distribution and collaboration seems like a great strategy, not just for us active in the business, but for the enterprise as well.
Prenumerera på:
Inlägg (Atom)