Title sums it up. I would just try it myself but my test environment is down and I figured I might get a speedier response here than to wait for that to come up.
If users don't have permissions to read an Excel file uploaded to a document library, can they still view a Chart Web Part that they are given permissions to use?