There are 2 things to do to approach this.
First you need to get your source folders into a build centric layout, this will help to eliminate as much overlapping as possible.
If you need a particular shared folder that shouldn't trigger a build, then don't include it in the source mappings, instead add a script to download the files to your workspace as an early part of the build.
The example will need updating for your visual studio version, and you should pass the sources Directory to the script.
REM %1 represents the Sources directory
REM Compute variables
SET TfExe="%ProgramFiles(x86)%\Microsoft Visual Studio 10.0\Common7\IDE\TF.exe"
REM SET TfExe="C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\TF.exe"
Set RefPath="$/TFS BUILDS/Shapes/Main/Includes"
Set localPath="%~1\Includes"
REM set the Drive Letter for this build
Set Localdrive=%localPath:~1,2%
%Localdrive%
cd %1
REM Map the folders
%TfExe% workfold /map %RefPath% %localPath%
REM Get the required content
%TfExe% get %RefPath%
REM Unmap the folders
%TfExe% workfold /unmap %RefPath%