30 November, 2012

Query SharePoint List to get all document versions information: SharePoint 2007

Two days back I have posted an article on SharePoint Storage Problems, which can give you a fair idea of what things to look at when you plan or your SharePoint site has eaten space more than expected.

While looking at the same problem, we already know that SharePoint document versions can be the one of the element of sudden spike in SharePoint Storage but how we could determine how many versions are there on the SharePoint collection and what size they are off?

As you all know SQL is the ocean for the SharePoint Information storage, so SQL can provide us the relevant information about this. I wrote a SQL query to get the information on the versioning of all the document in the particular site.

Select a.version, a.timecreated, a.size, b.dirname,b.leafname, b.listid
from dbo.DocVersions a, alldocs b  where a.siteid=b.siteid
and b.dirname = 'sites/AMAR' and a.size 'greator than and equal to' (<>);0 and b.listid is not null
order by a.size

In the above query, columns refer to:
Versions: refer to number of versions of documents in a list
Timecreated: time when the document was created
Size: is the size of the versions (in KB)
Dirname: site name (e.g. sites/amar)
List id: ID of the list in the SharePoint Site.

Hope this piece of information does help you.

If you have any questions and concerns do let me know.

28 November, 2012

SharePoint List Query Designer (Report Builder)

Report Builder provides both a graphical query designer and a text-based query designer to help you create a query that specifies the data to retrieve from a SharePoint site for a report dataset. Use the graphical query designer to explore the SharePoint list metadata, interactively build a query, and view the results of your query. Use the text-based query designer to view the query that was built by the graphical query designer, modify a query, or type the query commands. You can also import an existing query from a file or report.

In the graphical query designer you can explorer the SharePoint site, interactively build the command that retrieve SharePoint list data for a dataset. You choose the fields to include in the dataset and optionally, specify filters that limit the data in the dataset. You can specify that filters are used as parameters and provide the value of the filter at run-time.

SharePoint lists include a large number of SharePoint specific fields that might not be useful to include in reports. The query designer provides an option to hide these fields to make it easier and quicker to determine the fields to use.

To find more details about it, Click here.

26 November, 2012

SharePoint Storage Problems: MOSS 2007, SharePoint 2010, SharePoint 2013

Most SharePoint storage concerns are about size calculations. You will find some information below on how disk space is occupied by the site and how its content is calculated.  

First of all, adding a document to a library consumes much more space than you might think. For every document, there is also the metadata and index data that goes with it. If a document is stored in a standard folder and has a standard set of properties associated with it, the document will consume about 12 KB for metadata, plus about 30% of the total document size for indexing. Therefore, if a document is 100 KB in size, you can expect to consume 100 KB for the document, 12 KB for the metadata, and 33.3 KB for the index. Your 100 KB document just consumed 145 KB.

Other common reasons for unexpected site growth are the use of versioning in Document Libraries or an uncontrolled Second Stage Recycle Bin.

Problem #1: Versioning in Document Libraries

Versioning creates a new copy of the document every time the document has been checked out. Therefore, to determine how much space a document in an enhanced folder is consuming, take the number of versions plus one and multiply it by the document size plus the index size, plus the metadata size. This means that if there were four versions of our 100 KB sample document, the document would be consuming 725 KB (four versions plus 1, multiplied by the 100 KB document size, plus 33.3 KB for the index size plus 12 KB of metadata).

Solution: To control versioning settings, open the library and navigate to Settings &gt; Document Library Settings. Then click  Versioning Settings. To learn about versioning, refer to Help for SharePoint on the Microsoft site: http://office.microsoft.com/en-us/sharepointtechnology/HA100215761033.aspx

Problem #2: Site Collection Recycle Bin (or Second Stage Recycle Bin)


Whenever the end user deletes data from the site, it goes to the End User Recycle Bin. Items in the Recycle Bin remain there until the end user decides to permanently delete or restore them, or until the items are permanently deleted after the retention period of 30 days. To view items deleted from the site, a user can click the Recycle Bin link in the Quick Launch pane on the left. Users will see only their own files that have been deleted from the site. Even the site Administrator will see only his or her own deleted files in the first-level Recycle Bin.

Solution: If the file is deleted from the "user" Recycle Bin, it is sent to the Site Collection Recycle Bin, where an administrator can restore it or delete it permanently. The Second Stage Recycle Bin can be managed only by a SharePoint Administrator (a user with an email address that you specified as administrator during SharePoint installation).
To view the content of the Site Collection Recycle Bin:
1.      Log in with site administrator credentials.
2.      Navigate to Site Actions &gt; Site Settings.
3.      Under Site Collection Administration, click the Recycle Bin link.
This opens the Site Collection Recycle Bin, which has two links in the navigation pane on the left:
Problem #3: Unstructured data takeover.
The primary document types stored in SharePoint are PDFs, Microsoft Word and PowerPoint files, and large Excel spreadsheets. These documents are usually well over a megabyte.
SharePoint saves all file contents in SQL Server as unstructured data, otherwise known as Binary Large Objects (BLOBs). Having many BLOBs in SQL Server causes several issues. Not only do they take up lots of storage space, they also use server resources.
Because a BLOB is unstructured data, any time a user accesses a file in SharePoint, the BLOB has to be reassembled before it can be delivered back to the user – taking extra processing power and time.
Solution: Move BLOBs out of SQL Server and into a secondary storage location – specifically, a higher density storage array that is reasonably fast, like a file share or network attached storage (NAS).
Problem #4: An avalanche of large media.
Organizations today use a variety of large files such as videos, images, and PowerPoint presentations, but storing them in SharePoint can lead to performance issues because SQL Server isn't optimized to house them.
Media files, especially, cause issues for users because they are so large and need to be retrieved fairly quickly. For example, a video file may have to stream at a certain rate, and applications won't return control until the file is fully loaded. As more of this type of content is stored in SharePoint, it amplifies the likelihood that users will experience browser timeout, slow Web server performance, and upload and recall failures.
Solution: For organizations that make SharePoint “the place” for all content large and small, use third-party tools specifically designed to facilitate the externalization of large media storage and organization. This will encourage user adoption and still allow you to maintain the performance that users demand.
Problem #5: Old and unused files hogging valuable SQL Server storage.
As data ages, it usually loses its value and usefulness, so it’s not uncommon for the majority of SharePoint content to go completely unused for long periods of time. In fact, more than 60 to 80 percent of content in SharePoint is either unused or used only sparingly in its lifespan. Many organizations waste space by applying the same storage treatment for this old, unused data as they do for new, active content, quickly degrading both SQL Server and SharePoint performance.
Solution: Move less active and relevant SharePoint data to less expensive storage, while still keeping it available to end users via SharePoint. In the interface, it helps to move these older files to different parts of the information architecture, to minimize navigational and search clutter. Similarly, we can “unclutter” the storage back end.
A third-party tool that provides tiered storage will enable you to easily move each piece of SharePoint data through its life cycle to various repositories, such as direct attached storage, a file share, or even the cloud. With tiered storage, you can keep your most active and relevant data close at hand, while moving the rest to less expensive and possibly slower storage, based on the particular needs of your data set.
Problem #6: Lack of scalability.
As SharePoint content grows, its supporting hardware can become underpowered if growth rates weren't accurately forecasted. Organizations unable to invest in new hardware need to find alternatives that enable them to use best practices and keep SharePoint performance optimal. Microsoft guidance suggests limiting content databases to 200GB maximum unless disk subsystems are tuned for high input/output performance. In addition, huge content databases are cumbersome for backup and restore operations.
Solution: Offload BLOBs to the file system – thus reducing the size of the content database. Again, tiered storage will give you maximum flexibility, so as SharePoint data grows, you can direct it to the proper storage location, either for pure long-term storage or zippy immediate use.
It also lets you spread the storage load across a wider pool of storage devices. This approach keeps SharePoint performance high and preserves your investment in existing hardware by prolonging its useful life in lieu of buying expensive hardware. It’s simpler to invest in optimizing a smaller SQL Server storage core than a full multi-terabyte storage footprint, including archives.
Problem #7: Not leveraging Microsoft’s data externalization features.
Microsoft’s recommended externalization options are Remote BLOB Storage (RBS), a SQL Server API that enables SharePoint 2010 to store BLOBs in locations outside the content databases, and External BLOB Storage (EBS), a SharePoint API introduced in SharePoint 2007 SP1 and continued in SharePoint 2010.
Many organizations haven't yet explored these externalization capabilities, however, and are missing out on significant storage and related performance benefits. However, native EBS and RBS require frequent T-SQL command-line administration, and lack flexibility.
Solution: Use a third-party tool that works with Microsoft’s supported APIs, RBS, and EBS, and gives administrators an intuitive interface through SharePoint’s native Central Administration to set the scope, rules and location for data externalization.
In each of these five problem areas, you can see that offloading the SharePoint data to more efficient external storage is clearly the answer. Microsoft’s native options, EBS and RBS, only add to the complexity of managing SharePoint storage, however, so the best option to improve SharePoint performance and reduce costs is to select a third-party tool that integrates cleanly into SharePoint’s Central Administration. This would enable administrators to take advantage of EBS and RBS, choosing the data they want to externalize by setting the scope and rules for externalization and selecting where they want the data to be stored.
If you have any difficulties in understanding the above, please do let me know.

25 November, 2012

Upgrade process from SharePoint 2010 to SharePoint 2013

To upgrade from SharePoint 2010 Products to SharePoint 2013, you use the database-attach method to upgrade. In the database-attach method, you first create and configure a SharePoint 2013 farm. Then you copy the content and service application databases from the SharePoint 2010 Products farm, and then attach and upgrade the databases. This upgrades the data to the new version. Site owners can then upgrade individual site collections.

TechNet Article: http://technet.microsoft.com/en-us/library/cc262483.aspx
Upgrading from SharePoint 2010 to SharePoint 2013 Step by Step:
This is a step by step guide on upgrading a SharePoint 2010 site to a SharePoint 2013 (Customer Preview) and as with any Beta release is subject to change. For further information on planning and upgrading content and services to SharePoint 2013 you can visit the recently published TechNet site here:
Also make sure to check out Bill Baer's great post on Upgrade, and special thanks to Bill for sharing his custom solution used for this post.
1.      Add and deploy any required O14 custom solutions in your SharePoint 2013 farm, notice that all O14 solutions being deployed under 14 hive but eventually it's recommended to migrate the custom code to O15.
2.      Backup and restore the SharePoint 2010 content database to your SQL Server hosting SharePoint 2013 farm.
3.      Run Test-SPContentDatabase cmdlet to identify missing components along with potential errors and related warnings. Check the upgrade log and deploy any missing components and re run the cmdlet to verify.
4.      Attach the content database to the desired web application using Mount-SPContentDatabase cmdlet.
5.      After successfully mounting the content database to web application, the site should be accessible in 14 mode.
6.      To upgrade the site simply click on "Start now" link on the toolbar, you can also go to SiteUpgrade page from Site Setting page as well.
7.      Click on "TRY A DEMO UPGRADE" link to verify the site collection upgrade (This step can be performed by Site Collection Administrators, farm admins can run
Request-SPUpgradeEvaluationSiteCollection cmdlet as well).
8.      Provision a temporary site collection to validate the site post-upgrade.
9.      An email should be sent out to Site Collection admin when the temporary site is provisioned.
10.      After validating the temporary site we upgrade the site by going to SiteUpgrade.aspx page and clicking on "Upgrade the Site Collection" button (Farm admins can run Upgrade-SPSite cmdlet as well).
11.      During the upgrade The SiteUpgrade.aspx page shows the progress and provides a link to an upgrade log for troubleshooting purposes.
12.  Your site collection should now be accessible in 15 mode along with all new capabilities enabled in the farm and for the hosting web application such as Social and Office Web Apps.


Curtsey: Zillions Thanks to Ali Mazaheri for his explanatory Post.
If you have any issue during the upgrade do let me know. I would be my pleasure to work for you.