We noticed a performance oddity with our ETLs and figured out a solution, but I wanted to bring it to Blackbaud's attention for consideration of remediation in a future service pack.

We realized that Incremental ETLs, for Smart Fields, were significantly slower than Full ETLs. Since Smart Fields don't have a DATECHANGED field, we assumed they would always run as a Full ETL and shouldn't be significantly different. When we dug into it, we realized that while a Full ETL truncates the SmartField tables before filling them, an Incremental deletes the values for each smart field as it iterates through them. This is WAY slower. We cloned the OOB Constituent and Revenue Smart Field packages and added a truncation step at the beginning. For our largest affiliate, it dropped their ETL time from 3.5-4 hours to 1.5 hours. Our our smaller affiliates, it has similar impact - just not as dramatic in time saved. We've not come up with a scenario where the OOB behavior is needed or preferred - the net results seems to be the same. Thank you.

We've cloned and modified the OOB ETL packages for smart fields. We're fine for now, but it would be nice if the change were baked into the OOB versions.