Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Expiring API Keys #3092

Closed
harikmenon opened this issue Jun 22, 2016 · 41 comments
Closed

Support Expiring API Keys #3092

harikmenon opened this issue Jun 22, 2016 · 41 comments
Assignees

Comments

@harikmenon
Copy link

Details available here

http://blog.nuget.org/20160622/NuGet-API-key-expiration.html#comment-2745173305

@harikmenon harikmenon self-assigned this Jun 22, 2016
@harikmenon harikmenon added this to the Backlog milestone Jun 22, 2016
@harikmenon
Copy link
Author

@yishaigalatzer

@tugberkugurlu
Copy link
Contributor

I think this is overdoing it. It should be opt in, not mandated. Take the GitHub example. Allow generating different tokens and giving them a context. Make it possible to revoke the tokens on demand.

@mythz
Copy link

mythz commented Jun 22, 2016

Adding the ability to regenerate different API Keys is fine but breaking existing automated systems should never be the default behavior.

As mentioned on the thread there are more sane security options like an IP Whitelist, or allowing multiple API keys per account scoped to different packages. Or if you're concerned about stale packages you can revoke keys that haven't been used in a year. But whatever you decide, do not break existing systems or at the very least provide an option to opt-out entirely.

@vcsjones
Copy link

vcsjones commented Jun 22, 2016

Copied from comment on blog

OK. I seem to understand the changes you're making. Let's see the reasoning.

Making your API key on NuGet.org more secure is one more step in that direction.

Hmm. That's not much to go on. Let's assume the MO is "better security" since that seems to be the entirety of what the reason is.

I'll summarize by saying that a lot of people's concerns are due to the use of keys in automated environments. These environments that are supposed to be fully autonomous now require manual intervention every 90 days, or 4 times a year. This manual intervention is an action of privilege. Someone that has the nuget credentials must generate a new API key and update their configuration. Ironically, those that took the most trouble to secure and lock down their API key the most will likely have the most difficulty updating it. This change punishes those that were doing that, and drives a mentality of "well if it's going to change every 90 days I guess I won't worry about it too much". Those that encrypted it, locked down ACLs, etc, are now faced with a troublesome task.

Furthermore, those that did take care to lock things down may have done so in a way that the general ops / development team couldn't even do. Now, since we know the API key is going to expire, I might be inclined to open up the API key access so that anyone in ops / dev can update it, including sharing the nuget credentials used to make the API keys in the first place.

It's difficult to say whether or not this increases security without knowing what the risks are. A few that I can think of:

Leaked Access Key: This can either be unintentional, such as an accidental inclusion in a GitHub repository. It can also be "intentional" insomuch that the person intentionally committed it, not fully aware of the power it provides. In the former case, rotating keys helps marginally. Leaked keys are usually scraped in minutes, not days, because it's easy to automate. The latter can't really be helped. If they need to change it every 90 days, they'll just keep committing it in the open. In either case, this warrants immediate action, such as account suspension or revocation of all keys.

Staff turn over is another. A person may have taken a photo of it with their phone so they wouldn't have to look it up, written it down, etc. Ideally, an organization would immediately rotate the API key after the employee left. Changing every 90 days helps reduce the risk of the API key being in the open indefinitely. In this case, I agree that changing helps.

Since Let's Encrypt has been brought up, there is a good argument for 90 day certificates: it encourages people to completely automate the process. There is no such automation here, it's an entirely manual process that is error prone. Let's Encrypt cert-bot does all of the work for the user after a one-time initial registration.

Finally, there is the issue of choice. This will be entirely forced, without a concise explanation of what the security risks are.

@anaisbetts
Copy link

anaisbetts commented Jun 22, 2016

As @tugberkugurlu mentioned, this should work exactly like GitHub personal OAuth tokens, specifically:

  1. Give tokens semantic names, so that you can...
  2. ...make tokens one-time visible like GitHub does (i.e. no longer visible once page refreshes)
  3. Allow users to revoke them
  4. Notify the user via Email that a token has been created.

Take all of the existing keys and grandfather them as "NuGet Legacy Key", and let users optionally revoke them too

@davidwengier
Copy link

The package I maintain is niche market, not frequently updated, and is mainly a way for our company to "give back" to the community instead of keeping everything in house. Since the cadence of updates for the package is almost never going to be more frequent than 90 days, it means any time any change happens we would have to generate a new API key. This effectively nullifies any CI automation we have, and makes the process manual, always and forever.

I worry that that kind of friction will have a big impact on peoples choice to adopt, or possibly even continue using, this platform.

@motowilliams
Copy link

If the expiry behavior is an absolute must then give Refresh Tokens a look https://tools.ietf.org/html/rfc6749#section-1.5 so automated systems could at least be updated to handle this new behavior.

@JamesNK
Copy link

JamesNK commented Jun 22, 2016

This sounds like a pain in the butt. +1 for @paulcbetts approach.

@ferventcoder
Copy link
Contributor

Package signing (likely GPG/PGP) is how the Chocolatey team is looking at providing authenticity and traceability of packages. We are not really looking at doing this kind of thing, for most of the reasons stated all over the comments.

We also do moderation of the gallery/community package repository and VirusTotal checking of all packages.

@Oceanswave
Copy link

I was going to put forth my opinion, but my GitHub 90-day comment key has expired so meh; I'll just keep it to myself. It probably wasn't that good of an opinion anyway.

@yishaigalatzer
Copy link

@ferventcoder moderation is not something that is achievable for nuget with thousands of packages being pushed every day.

Signing is like mentioned above on our roadmap as well.

Virus scanning of all packages is already in for quite some time (both every new package gets scanned, and the whole list is scanned regularly).

@SimonCropp
Copy link

@yishaigalatzer where is the virus scanning documented?

@yishaigalatzer
Copy link

@SimonCropp it is running as an internal Microsoft service, not part of the main project.

@SimonCropp
Copy link

@yishaigalatzer ok but it would be nice if somewhere in the doco it said "we scan packages and this is the scanning suite we use"

@yishaigalatzer
Copy link

We can say we do CC @harikm86 (can you please add?)
The suite is an internal Microsoft tool, so nothing meaningful we can report.

@SimonCropp
Copy link

@yishaigalatzer so was there any communication at all that nuget does virus scanning? blog post perhap? just trying to understand how i missed it

@ferventcoder
Copy link
Contributor

I also did not know this about virus scanning. I figured it was likely, given NuGet is quite so well received.

@ferventcoder
Copy link
Contributor

@yishaigalatzer I figured moderation would be difficult given the scale for NuGet. Libraries rev quicker than app releases. For us it is around 100-500 per week.

@yishaigalatzer
Copy link

We are planning self approvals as part of the staging feature (so someone can upload, but someone else approve), not so much moderation but in a way can enforce another set of eyes for teams.

@gulbanana
Copy link

there are many stable nuget packages which are updated less frequently than 90 day intervals. it is not a good idea to prevent automation and CI for this category of software.

@anaisbetts
Copy link

anaisbetts commented Jun 23, 2016

We are planning self approvals as part of the staging feature (so someone can upload, but someone else approve), not so much moderation but in a way can enforce another set of eyes for teams.

Let me get on my pulpit for a minute. This all sounds like a ton of developer and QA work, creating feature after feature that fundamentally put more obstacles in the way of users sharing code with the larger community, a community that is already struggling to get people to contribute. And here's a place where I see a huge opportunity cost.

How much better would the .NET world be if all of that energy was instead put towards making it drop-dead easy to both create and contribute to Open Source. Why is creating a NuGet package not part of every new "Windows Library" template? Why can't I right-click on any package and "Fork package from GitHub", then send a Pull Request back?

If y'all put your heads together, you could think of dozens of good ideas for how NuGet could be promoting its userbase to contribute and improve our community, rather than be silent consumers! At the end of the day, we end up with a way more exciting and fun future in the .NET World.

@yishaigalatzer
Copy link

Creation is a key tenet for the next couple of releases. You have seen part of it with xproj, and we are investing more on multiple fronts for improving the authoring experience.

There are several investments in basic infrastructure that will fix the caching model to allow for easily allowing to override packages.

At the same time security is a key tenet for nuget.org and we are going to invest in it heavily. This particular feature im describing might not fit your needs but it does fit other teams needs that have stricter requirements in their publishing process. This is planned as an optional feature, not as something that will get in your way.

Btw I'll love to talk to you on slightly related but not relevant to this thread subject. Feel free to hit me on my email yigalatz @ msft

@harikmenon
Copy link
Author

Adding a bit more to what Yishai mentioned. That is valid feedback and you will see us investing more in creation and contribution going forward. The feature that you mentioned, around being able to build a package right of a project is something we are designing and working on now with Xamarin.

We need to balance feature requests, asks and aspirations of both users of packages and authors. From an authoring perspective, creation and contribution are central tenets which I completely agree with, while for consumers of packages other priorities like being able manage dependencies effectively across components, being able trust packages that they consume are paramount.

@yishaigalatzer
Copy link

Good one

Get Outlook for Androidhttps://aka.ms/ghei36

On Wed, Jun 22, 2016 at 9:06 PM -0700, "Harikrishna Menon" <notifications@github.commailto:notifications@github.com> wrote:

Adding a bit more to what Yishai mentioned. That is valid feedback and you will see us investing more in creation and contribution going forward. The feature that you mentioned, around being able to build a package right of a project is something we are designing and working on now with Xamarin.

We need to balance feature requests, asks and aspirations of both users of packages and authors. From an authoring perspective, creation and contribution are central tenets which I completely agree with, while for consumers of packages other priorities like being able manage dependencies effectively across components, being able trust packages that they consume are paramount.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//issues/3092#issuecomment-227944713, or mute the threadhttps://github.com/notifications/unsubscribe/ABLmt2qb8T4jSZqwVLuYQevmoygSIdOPks5qOgZQgaJpZM4I8Nl-.

@MichaelKetting
Copy link

It's already been said several times on this thread and the blog post, but just in case this gets to be a vote ;)

The API Key regeneration is a painful process that will then have to be performed 5 (!) times a year, unless you want to create a last-minute calendar entry or just wait until the next release build fails. For me, I use a dedicated build user, so I need to start a new browser isntance (unless I want to log out of my microsoft account on my main browser), dig out the password, log in to nuget.org, regenerate the API key and then I can go and copy & paste it into the build configuration.

While I know that I can sustain this process, it will certainly cause some annoyance on an 80 day recurrance. I don't believe there should be annoyance in an automated system.

@yishaigalatzer
I'm curious how the key is handled in dotnet core / asp.net vNext? I'm certain Microsoft's own package publishing will face similiar issues?

On the blog thread there was a suggestion of fine grained permissions and tokens. I would most definitely love to have this option so I can generate a token that is only allowed to publish packages, not overrite them, etc.

@yishaigalatzer
Copy link

Tokens can only be used to publish packages, or unlist them. Overwriting was never allowed.

The suggestion was to associate tokens to specific packages rather than just to an owner.
Microsoft packages were going to get even shorter token lifetime (1 day) and extra internal scrutiny.

@MichaelKetting
Copy link

Thanks for the quick response. Thanks for clearing up the token bit.

1 day? Wow. I guess, since new versions will not get releases on a daily basis , you can afford to have someone update the tokens during a release. Or, I guess the correct question should be, do you do automated uploads to nuget or are they a manual process?

Hmm... I just had an idea: What about client certificates in addition to tokens? That way, you could lock the upload during an automated build to a specific build agent and user account. And for additional security, you could use a physical machine and put the certificate on a usb token. Sure, such a setup would not scale for just any random package but for high profile stuff, it would really lock things down.

The alternative is pacakge signing and actually locking down nuget to only accept a new version of a package if the public key matches the one registered for this package. I think I like that option the most, actually, since having a secure signing setup is already part of a secure publication process.

@moswald
Copy link

moswald commented Jun 23, 2016

This is planned as an optional feature, not as something that will get in your way.

Which part is optional - the moderation, or the 90-day expiration? I'm with Paul on this one, if the 90-day expiration isn't at best opt-out, it will drive everyone away - including any customers who are currently asking for this kind of security feature. It will become a barrier to entry that most simply won't bother with.

@harikmenon
Copy link
Author

@moswald : The moderation using staging is optional

@asbjornu
Copy link

I'm with @paulcbetts on everything he writes. I urge the NuGet team to go back to the drawing board on this one and don't do anything about this issue before something as good as GitHub's OAuth keys have been implemented.

While the currently proposed change is done with the best intentions and improved security is something everyone wants, this is not going to work. Auto-expiry without a way to auto-refresh is not going to give better security, since it will force people to automate the login and click-through of generating the API key, possibly exposing passwords, which is much worse.

Security measures that disregard convenience and user experience eventually hurts security more than it improves it.

@forki
Copy link
Contributor

forki commented Jun 23, 2016

Such script already exists. Which proves the point.
On Jun 23, 2016 7:53 PM, "Asbjørn Ulsberg" notifications@github.com wrote:

I'm with @paulcbetts https://github.com/paulcbetts on everything he
writes. I urge the NuGet team to go back to the drawing board on this one
and don't do anything about this issue before something as good as GitHub's
OAuth keys have been implemented.

While the currently proposed change is done with the best intentions and
improved security is something everyone wants, this is not going to work.
Auto-expiry without a way to auto-refresh is not going to give better
security, since it will force people to automate the login and
click-through of generating the API key, possibly exposing passwords, which
is much worse.

Security measures that disregard convenience and user experience
eventually hurts security more than it improves it.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#3092 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AADgNBFsYxNBfCJUNA6vagvlbf10nR_Oks5qOsgmgaJpZM4I8Nl-
.

@NickCraver
Copy link
Contributor

As a contributor and maintainer of many packages, this is a very unwelcome change. If the behavior @paulcbetts expands on here (e.g. opt-in and grandfather) then I'm all for a new feature of control. Feature, not requirement. I wouldn't use it - but I see no fundamental problem with it, so long as perpetual keys are still allowed - even if they're not the default for new users. Depending on how tooling works though I think perpetual would have to be the default realistically, which does back to...how can the timed expiration really work at all?

What other service does this? I'm unaware of any service looking for participation (even including those with 2-factor authentication) that have such a manual renewal step. NuGet is also on the opposite end of the spectrum with a far higher percentage of the interactions being automated. We've tried extremely hard to eliminate manual interaction in all of these systems wherever possible.

Please, don't do this as planned. Pushing this as a breaking change will not go over well at all - look at the reactions on this thread, it's a very clear unified response from the community.

@harikmenon
Copy link
Author

Thanks @NickCraver. Appreciate the feedback.

@harikmenon
Copy link
Author

@SimonCropp I have opened an issue on docs for this https://github.com/NuGet/NuGetDocs/issues/479. do note that we currently in the process of refreshing this docs, so this change will only come in a moth or so once we had a chance to reorganize the docs a bit.

@gabrielweyer
Copy link

Even Azure allows users to create perpetual keys. At the end of the day it is the end user responsibility to store those credentials appropriately. If you still decide to go ahead with this change it should definitely be opt-in.

I understand you've to balance security and usability but this change introduces too much friction for teams that rely on automation.

I like @paulcbetts suggestions. Why can't we have multiple keys valid at the same time?

IP whitelisting is not an option as it would force to push from a known list of static IPs.

@mythz
Copy link

mythz commented Jun 25, 2016

IP whitelisting is not an option as it would force to push from a known list of static IPs.

IP whitelisting isn't supposed to be mandatory, it's an option for people who want to lock down publishing to a static IP address (ala static firewall rules), e.g. the CI server that publishes the packages.

@gabrielweyer
Copy link

What I meant is that build systems tend to be created on demand more and more and in this case static IP addresses are irrelevant.

AppVeyor and Travis both publish a list of public IPs but this means everyone else building there could bypass the IP filtering.

@mythz
Copy link

mythz commented Jun 25, 2016

It isn't applicable in any case where the publishers IP address isn't fixed or is shared, hence why it would be just another option, particular useful for anyone running their own CI servers.

@nblumhardt
Copy link

@harikmenon this thread seems to have gone silent - any chance of an update on this one?

@harikmenon
Copy link
Author

We will be publishing a blog post within the next 2 weeks with an update. BTW would love to reach out to you offline on something related. Can you shoot me an email at harikm@microosft.com

@harikmenon
Copy link
Author

Update on the changes to Expiring API Keys available here

http://blog.nuget.org/20160825/Changes-to-Expiring-API-Keys.html

@skofman1 skofman1 closed this as completed Jan 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests