I've heard this suggestion before, but I'm not so keen to use it.
First, because you only need to build the pods once (per Xcode/SDK change, that is), I prefer to check them all in. This approach does not work for node.js or other platforms, where the binary packages support a wide range of platforms.
Every time you switch branches, you have to run pod install/update, and wait for the pods to download. That's fine if you're having a proper internet connection, but does not work that well on ADSL or cell phone tethering (5GB data/month).
Also, because of the large file sizes, even if you add the ArcGIS.framework to your project, you still can't commit it to GitHub.
Currently, I've installed the SDK in ~/Library and will use this setup until the files size get to a reasonable value (or GitHub increases the file limit to 200M).