Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include build number when renaming hotfix releases of Unity #274

Closed
4 tasks done
JensRestemeier opened this issue Apr 8, 2018 · 21 comments
Closed
4 tasks done

Include build number when renaming hotfix releases of Unity #274

JensRestemeier opened this issue Apr 8, 2018 · 21 comments
Assignees

Comments

@JensRestemeier
Copy link

JensRestemeier commented Apr 8, 2018

Issue Checklist

  • I'm using the latest u3d version
  • I ran u3d --help
  • I read the README
  • I made sure that a similar issue doesn't exit

Issue Description

We sometimes have hotfix releases installed in parallel to official releases. They'll have the same version, but they'll have a different revision code. Renaming them to the plain version number will not be possible because of the conflict to the official release.

When I install them manually I usually use the Unity_<version>_<revision> pattern.

Suggestion:

  • check the installed revision to the official revision
  • if they match, use the usual Unity_<version> pattern
  • if they don't match, use Unity_<version>_<revision>

The official revision seems to be used as part of the download url, i.e.:

u3d available
...
Version 5.4.1f1: https://download.unity3d.com/download_unity/**649f48bbbf0f**/
...

This revision is displayed in Unity in Help/About after pressing "Alt" on the keyboard.

At the start of the log file:

Built from '5.4/release' branch; Version is '5.4.1f1 (649f48bbbf0f) revision 6594376'; Using compiler version '160040219'

Unfortunately I haven't found a good way to find this without running Unity, yet. On OSX it is in the UnityBuildNumber plist key. On Windows there seems to be a special "Unity Version" resource in the exe:

BLOCK "StringFileInfo"
{
BLOCK "040904b0"
{
VALUE "CompanyName", "Unity Technologies ApS"
VALUE "FileDescription", "Unity Editor"
VALUE "FileVersion", "5.4.1.6594376"
VALUE "LegalCopyright", "(c) 2016 Unity Technologies ApS. All rights reserved."
VALUE "ProductName", "Unity"
VALUE "ProductVersion", "5.4.1.6594376"
VALUE "Unity Version", "5.4.1f1_649f48bbbf0f"
}
}

This can be seen in "Resource Hacker". I am not a Ruby hacker, so I have no idea if there is a package to open .exe version resources...

@JensRestemeier
Copy link
Author

It would be useful if u3d looked for some kind of overrideVersion.txt file in a Unity installation, in case two copies of the same version are installed for other reasons. But that is less critical than the hotfix releases.

@lacostej
Copy link
Member

Hello @JensRestemeier Thanks for the feedback. This sounds like a feature for u3d. We'll look into it.

@lacostej
Copy link
Member

@lacostej
Copy link
Member

This code works for me:

require "Win32API"

GetFileVersionInfoSize = Win32API.new('version.dll', 'GetFileVersionInfoSize','PP','L')
GetFileVersionInfo = Win32API.new('version.dll', 'GetFileVersionInfo','PIIP','I')
VerQueryValue = Win32API.new('version.dll', 'VerQueryValue','PPPP','I')
RtlMoveMemory = Win32API.new('kernel32.dll', 'RtlMoveMemory', 'PLL', 'I')

def string_file_info(info, path)
  begin
      file = path.gsub(/\//,"\\")

      buf = [0].pack('L')
      version_size = GetFileVersionInfoSize.call(file + "\0", buf)
      raise Exception.new    if version_size == 0 # TODO use GetLastError

      version_info = 0.chr * version_size
      version_ok = GetFileVersionInfo.call(file, 0, version_size, version_info)
      raise Exception.new if version_ok == 0 # TODO use GetLastError
  
      # hardcoding lang codepage
      struct_path = "\\StringFileInfo\\040904b0\\#{info}"
	  
      addr = [0].pack('L')
      size = [0].pack('L')
      query_ok = VerQueryValue.call(version_info, struct_path + "\0", addr, size)
      raise Exception.new if query_ok == 0

      raddr = addr.unpack('L')[0]
      rsize = size.unpack('L')[0]
	  
      info = Array.new(rsize,0).pack('L*')
      RtlMoveMemory.call(info, raddr, info.length)
      info.strip
  rescue Exception => e
      #puts e
      #puts e.backtrace
      "N/A"
  end
end

FILENAME = "C:/Program Files/Unity/Editor/Unity.exe"
infos = ["CompanyName","LegalCopyright","ProductVersion","FileDescription","LegalTrademarks","PrivateBuild","FileVersion","OriginalFilename","SpecialBuild", # predefined VerQueryValue string-names
  "Unity Version", "FileVersion", "ProductName"] # Unity specific?
infos.each { |info| r = string_file_info(info, FILENAME); puts "#{info}: #{r}" }

# gives
%Q(
CompanyName: Unity Technologies ApS
LegalCopyright: (c) 2018 Unity Technologies ApS. All rights reserved.
ProductVersion: 2017.4.1.9581049
FileDescription: Unity Editor
LegalTrademarks: N/A
PrivateBuild: N/A
FileVersion: 2017.4.1.9581049
OriginalFilename: N/A
SpecialBuild: N/A
Unity Version: 2017.4.1f1_9231f953d9d3
FileVersion: 2017.4.1.9581049
ProductName: Unity
)

@niezbop
Copy link
Member

niezbop commented Apr 17, 2018

I just tested it locally, and it gave me the following:

CompanyName: Unity Technologies ApS
LegalCopyright: (c) 2017 Unity Technologies ApS. All rights reserved.
ProductVersion: 5.6.1.2646195
FileDescription: Unity Editor
LegalTrademarks: N/A
PrivateBuild: N/A
FileVersion: 5.6.1.2646195
OriginalFilename: N/A
SpecialBuild: N/A
Unity Version: 5.6.1f1_2860b30f0b54
FileVersion: 5.6.1.2646195
ProductName: Unity

@lacostej
Copy link
Member

And I was looking at the release notes for 2017.2.2 and found reference to this:

https://issuetracker.unity3d.com/issues/windows-standalone-player-exe-has-no-product-version-slash-file-version-in-properties

So some version of Unity don't have version information... This is going to be fun.

@JensRestemeier
Copy link
Author

That is the player executable, isn't it? That is the executable that is copied into the player folder to create a stand alone game, not the editor executable.
I kind-of hoped that inserting the build identifier into the version resources is part of their build process, not something that has to be remembered.

@JensRestemeier
Copy link
Author

I think Linux is going to need a custom solution, as there is no standard way to embed resources into a binary. There is the option to add custom sections to an elf file through the linker, but I have not idea if Unity does that, or which section they use... If you have a Linux version of the editor around you could use objdump or readelf to look for any custom sections. Otherwise you'd have to grep for the string in the .rodata section.

@lacostej
Copy link
Member

@JensRestemeier ah yes, it was the Player. Good.

For Linux we could go

$ strings /opt/unity-editor-2017.3.0f1/Editor/Unity | grep 2017.3.0f1
2017.3.0f1 (3c89f8d277f5)
2017.3.0f1_3c89f8d277f5
Invalid serialized file version. File: "%s". Expected version: 2017.3.0f1. Actual version: %s.
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.94 Safari/537.36 Unity/2017.3.0f1 (unity3d.com;
Unity 2017.3.0f1
UnityPlayer/2017.3.0f1 (UnityWebRequest/1.0, libcurl/7.51.0-DEV)

So this works

$ strings /opt/unity-editor-2017.3.0f1/Editor/Unity | ruby -ne 'print if /^[0-9\.fpb]+_[0-9a-f]{12}/'
2017.3.0f1_3c89f8d277f5

I could use readelf to make it more efficient.

@lacostej
Copy link
Member

@JensRestemeier I've added (as a PR) the foundation for detecting the build number. It will part of next release, feel free to test it.

Then will remain to design the feature set for using this. I'll make a proposal later.

@lacostej
Copy link
Member

I don't have a proposal. I have too many questions! To properly support this in all the features we need to look at the consequences / potential features:

  • list
    -> display build number [DONE]
    -> idea: display extra information? (LTS, hotfix, etc)

  • available
    -> idea: display extra information? (LTS, hotfix, etc). Would need to group them by channel sources, probably update unities.

  • install
    -> idea: take a URL, can be used to install hotfix releases
    if given a URL, install it as version_buildnumber?
    question: what if someone gives a regular version to be installed using its URL?
    question: how could you deal with all those extra URLs. Should we support something like
    u3d available add [URL|build_number]?

  • internals
    moving unities -> ouch! question: how do we know what to move to version vs version_build_number path?

  • u3d run / install. ProjectSettings/ProjectVersion.txt doesn't contain the hotfix version
    question: how do you select/install the needed version?

lacostej added a commit that referenced this issue Apr 19, 2018
u3d/list: display full revision number (prepares for #274)
@lacostej
Copy link
Member

I added a section on build numbers: https://github.com/DragonBox/u3d#unity-build-numbers

We will release 1.0.20 now and think about the proper full support for hotfixes in the coming weeks. Please test 1.0.20 when it is out! Thanks

@JensRestemeier
Copy link
Author

list:

  • not sure where extra info can come from. Maybe parsing release notes from the website? Hotfixes come normally because of a specific issue, which are later (after further QA on all platforms etc.) go into an official patch release.

install:

  • we got hotfix releases through a separate channel, so I don't think installation through u3d is easy or possible. The most important thing is not to get confused by multiple installed editors with the same version. Maybe when installing an official release it could be tagged as such, which means everything else is automatically a hotfix release.
    One option would be to allow installation from a local path or network share.

internals:

  • I think for most people unity_version will work best, and unity_version_build is only useful if you actually need to install multiple copies of Unity.

u3d run/install: For me both a runtime argument and a separate manifest file in the project folder would work.

@lacostej
Copy link
Member

@JensRestemeier question: are your hot fixes distributed through a hidden URL similar to this:
https://somedomain.unity3d.com/somepath/472613c02cf7/ ?

I.e. if I was feeding that URL to u3d install, would u3d be able to deal with it in the same way it deals with the official releases? (i.e. find the modules, etc). Or are those URLs hidden behind some user/password mechanism.

If the download URL is just hidden, we could do something like that

u3d install https://YOURHIDDENURL/BUILDNUMBER/
-> installs under /.../Unity_$version_$buildnumber/
-> add a special marker under the folder

When it comes to channels, I had code for supporting alpha releases as well. I haven't published it because it requires to manage credentials and wasn't sure many people would use it.

@JensRestemeier
Copy link
Author

It is a non-standard url scheme and password, at least for the ones we received. I am not sure if Unity would be happy if I went into more detail, and details may depend on which team handles the ticket.

@lacostej
Copy link
Member

@JensRestemeier understood.

So here's a proposal:

  1. add a (manual) new rename installation dir command

u3d mv VERSION|DIR_PATH DIR_NAME/DIR_PATH
-> moves a version into a particular directory
-> adds a .u3d_do_not_touch in $unity_install_dir

u3d mv -f|--force -l|--long VERSION|DIR_PATH

--long uses full path Unity_$version_$buildnumber
e.g. -> u3d mv 5.6.0f1 does same as u3d mv 5.6.0f1 Unity_5.6.0f1_123fbe67d

confirm y/n except if -f added or if a existing directory exists

  1. sanitize feature is changed so that it does not move installations that have a .u3d_do_not_touch file

  2. [later?] u3d mv --check_cache

for all the installed unities
identify the buildnumber
check if the cache has same buildnumber for that version on that platform
if no, rename the install as with long name

Does that sound good?

@JensRestemeier
Copy link
Author

Yes, that sounds great!

@JensRestemeier
Copy link
Author

By the way, I just had a thought how I'd implement version resources on Linux for a program, as there is no standard to do that. You can add custom sections to an ELF, but they are not standardised.
The common method is to embed a string with a special token that can be found with the strings tool.
One method is to convert a text file into an .o using objcopy, another is to generate a small .c file to include in the build. Both methods should generate an entry in the symbol table for the start of the data.

I don't have Linux running right now, or the Linux editor, or access to elf tools. It would be interesting to see if there is a symbol left to point at the start of the version string. That may be faster and safer to use than a general string search.

@lacostej
Copy link
Member

lacostej commented Apr 20, 2018

@JensRestemeier the implementation I made on Linux searches for the string in different places where Unity adds the string we are looking for. I found a few very small files and it takes a negliegeable amount of time (think 0.040s). I fall back on the Editor.

$ u3d console
>> require 'benchmark'
true
>> puts Benchmark.measure { U3d::LinuxInstallationHelper.new.find_build_number('/opt/unity-editor-2017.3.0f1') }
  0.020000   0.000000   0.040000 (  0.041639)
nil

@lacostej
Copy link
Member

@JensRestemeier I added a PR and briefly tested it.

I will need to clean it up quite a bit, but it shows the general idea. It looks like it worked.

For now I only support:

  • the .u3d_do_not_move
  • u3d move -f VERSION, which should cover your use case.

@JensRestemeier
Copy link
Author

Thank you, I'll have a look.
I remembered that I have access to a Linux server I can remote into, and I had a quick look. The strings are in the .rodata section, but don't seem to have a symbol nearby. Interestingly lots of third party libraries define symbols, for example sqlite3_version.

@lacostej lacostej self-assigned this Apr 20, 2018
lacostej added a commit that referenced this issue Apr 22, 2018
u3d/move command, renaming install dirs. Support long version names. Fixes #274
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants