long id

Oct. 1st, 2015 02:02 pm
juan_gandhi: (VP)
[personal profile] juan_gandhi
I think I got it where all this bs about passing around numerical ids of entities instead of entity references (maybe lazy) come from. It's like 'error code'. It comes from the ancient c programming, where we just could not allocate a string for a readable piece of text, or for the data that may need some efforts to instantiate or allocate.

In short. It's stupid to pass around "ids" in a program.

Date: 2015-10-01 09:10 pm (UTC)
From: [identity profile] rssh.livejournal.com
Hmm, is not 'persistent reference <=> id' ?

Date: 2015-10-01 09:44 pm (UTC)
From: [identity profile] juan-gandhi.livejournal.com
It is, but why pass it around if we can have a lazy data provider in memory and point to that memory; persistent reference hiding inside.

(no subject)

From: [identity profile] sassa-nf.livejournal.com - Date: 2015-10-01 10:09 pm (UTC) - Expand

(no subject)

From: [identity profile] juan-gandhi.livejournal.com - Date: 2015-10-02 12:27 am (UTC) - Expand

(no subject)

From: [identity profile] sassa-nf.livejournal.com - Date: 2015-10-02 06:49 am (UTC) - Expand

Date: 2015-10-01 09:25 pm (UTC)
dennisgorelik: 2020-06-13 in my home office (Default)
From: [personal profile] dennisgorelik
> It's stupid to pass around "ids" in a program

What if you are passing "ids" to database (to retrieve full records)?
Is it still stupid?

Date: 2015-10-01 09:45 pm (UTC)
From: [identity profile] juan-gandhi.livejournal.com
That's technical. ids are like addresses. You don't have to see them.

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-01 10:36 pm (UTC) - Expand

(no subject)

From: [identity profile] juan-gandhi.livejournal.com - Date: 2015-10-02 12:28 am (UTC) - Expand

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-02 01:00 am (UTC) - Expand

(no subject)

From: [identity profile] exceeder.livejournal.com - Date: 2015-10-02 01:31 am (UTC) - Expand

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-02 01:49 am (UTC) - Expand

(no subject)

From: [identity profile] exceeder.livejournal.com - Date: 2015-10-02 02:11 am (UTC) - Expand

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-02 02:54 am (UTC) - Expand

(no subject)

From: [identity profile] yatur.livejournal.com - Date: 2015-10-02 06:41 am (UTC) - Expand

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-02 09:56 am (UTC) - Expand

(no subject)

From: [identity profile] exceeder.livejournal.com - Date: 2015-10-02 07:08 am (UTC) - Expand

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-02 10:02 am (UTC) - Expand

(no subject)

From: [identity profile] exceeder.livejournal.com - Date: 2015-10-02 05:18 pm (UTC) - Expand

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-02 08:33 pm (UTC) - Expand

(no subject)

From: [personal profile] dennisgorelik - Date: 2015-10-02 02:04 am (UTC) - Expand

(no subject)

From: [identity profile] exceeder.livejournal.com - Date: 2015-10-02 02:28 am (UTC) - Expand

Date: 2015-10-01 09:26 pm (UTC)
From: [identity profile] vit-r.livejournal.com
Когда количество перерастает в качество и таблица перестаёт помещаться в памяти сервера, начинаются интересные вещи.

Потом ещё великолепны будут способы вычисления уникальности.

Date: 2015-10-01 09:43 pm (UTC)
From: [identity profile] rezkiy.livejournal.com
< rant>

so what do you do if you actually DO fail to allocate? you pass an exception through whatever means. Let's now talk about how your cyclomatic complexity affects your ability to reason about correctness of your program.

If you never learnt to code in imperative programming languages (such as C/C++), it doesn't exactly mean that those that did learn are stupid or something.

< /rant>

Date: 2015-10-01 09:46 pm (UTC)
From: [identity profile] juan-gandhi.livejournal.com
You don't have to allocate everything for everything. Conceptually it's here; actually it's not, and all we have, encapsulated, is that id. But we don't want to see it.

They are not stupid. They are just from an old age. mid-XX century.
Edited Date: 2015-10-01 09:46 pm (UTC)

(no subject)

From: [identity profile] rezkiy.livejournal.com - Date: 2015-10-01 09:55 pm (UTC) - Expand

(no subject)

From: [identity profile] juan-gandhi.livejournal.com - Date: 2015-10-02 12:30 am (UTC) - Expand

(no subject)

From: [identity profile] rezkiy.livejournal.com - Date: 2015-10-02 01:03 am (UTC) - Expand

(no subject)

From: [identity profile] gineer.livejournal.com - Date: 2015-10-02 07:36 am (UTC) - Expand

(no subject)

From: [identity profile] metaclass.livejournal.com - Date: 2015-10-02 09:52 am (UTC) - Expand

Date: 2015-10-01 09:47 pm (UTC)
From: [identity profile] no more turtles (from livejournal.com)
Имеет ли эта мудрость отношение к тому, что юниксовский сервер под полным лоадом с аптаймами больше года это в порядке вещей, а жабовский сервер без особой загрузки мне приходится бегать и перегружать каждую неделю, потому что OOM exception и вся машина висит как зомби (да, вся, жаба неубиваема!), хотя и на пинг реагирует.

Date: 2015-10-02 12:33 am (UTC)
From: [identity profile] juan-gandhi.livejournal.com
Конечно имеет. Открываем на телефоне шелл и телнетим на сервер, чтобы жужу почитать в аски. Желательно в виде номеров анекдотов.

(no subject)

From: [identity profile] no more turtles - Date: 2015-10-02 05:02 pm (UTC) - Expand

Date: 2015-10-01 11:36 pm (UTC)
From: [identity profile] cema.livejournal.com
Yes, it's dome more C legacy. But we do need a handle, just a better abstraction than a number.

Date: 2015-10-02 12:33 am (UTC)
From: [identity profile] juan-gandhi.livejournal.com
Sure. It does not matter how we retrieve values via pointers.

Date: 2015-10-02 12:07 am (UTC)
From: [identity profile] soonts.livejournal.com
>passing around numerical ids of entities instead of entity references
Numerical ids are much better.

>It's like 'error code'.
No it’s not.

You can store them in a CPU register.
You can save them using only 8 bytes of storage.
You can compare them with a single cmp CPU instruction.
You can design your system so the IDs are unique even if your system will scale out to thousands of servers.

Date: 2015-10-02 12:36 am (UTC)
From: [identity profile] juan-gandhi.livejournal.com
Oh shit, never thought about CPU registers somewhere inside my phone.

And I never suggested to pass around internal references outside. This may be an interesting problem, passing around internal references; even a numerical id is an internal reference if we look from outside. I routinely talk to our analysts like this - "did it work now for account number 10567"? Now imagine... well, we all know how it is. Phone number (why the fuck it is still a number?), SSN, card number (why number?), etc.

(no subject)

From: [identity profile] rezkiy.livejournal.com - Date: 2015-10-02 01:08 am (UTC) - Expand

(no subject)

From: [identity profile] soonts.livejournal.com - Date: 2015-10-02 01:28 am (UTC) - Expand

(no subject)

From: [identity profile] juan-gandhi.livejournal.com - Date: 2015-10-02 05:00 am (UTC) - Expand

(no subject)

From: [identity profile] sassa-nf.livejournal.com - Date: 2015-10-02 07:09 am (UTC) - Expand

(no subject)

From: [identity profile] yatur.livejournal.com - Date: 2015-10-02 04:45 am (UTC) - Expand

(no subject)

From: [identity profile] juan-gandhi.livejournal.com - Date: 2015-10-02 04:56 am (UTC) - Expand

(no subject)

From: [identity profile] sassa-nf.livejournal.com - Date: 2015-10-02 07:07 am (UTC) - Expand

Date: 2015-10-02 04:32 am (UTC)
From: [identity profile] yatur.livejournal.com
> You can store them in a CPU register.
> You can save them using only 8 bytes of storage.
> You can compare them with a single cmp CPU instruction.

These are all "C" concerns. The difference is that for Vlad "C" language (and a CPU register) sounds like something from the age of pyramids, while for embedded developers this is painful present.

> You can design your system so the IDs are unique

This does not really say something in favor or against passing "naked" IDs vs lazy entities. You can pass around lazy entities with unique IDs.

Edited Date: 2015-10-02 04:33 am (UTC)

Date: 2015-10-02 03:59 am (UTC)
From: [identity profile] yatur.livejournal.com
I think C programming has nothing (or little) to do with it. Access to (relational) databases via lazy objects references is very harmful to performance.

Databases are designed to get mass quantities of records in one shot. E.g. retrieving 10,000 records via some simple query (SELECT * FROM Users WHERE OrganizationId=125) is virtually instantaneous. Lazy entity references are designed to get one object at a time. Retrieving 10,000 records via 10,000 queries (SELECT * FROM Users WHERE UserId=n x10,000 times) will take forever.

ORMs like Hibernate do provide lazy references, and make things look easy on toy databases. However, this ease is deceiving. Later on you find yourself rewriting half of the program getting rid of the lazy references, because it just does not work fast enough in production.
Edited Date: 2015-10-02 04:03 am (UTC)

Date: 2015-10-02 06:27 am (UTC)
From: [identity profile] sleepy-drago.livejournal.com
You saying it like ppl dont know about "Later on...". Pro know exactly when to cash their stock and move on from a startup to greener pastures.

C programmers have other sins. Often they mix numbers that are identifiers for different things. I once coined for myself a label after working some legacy codebase: "program composed from integers". If highest abstraction in a code you have to ship is integer you know you're screwed.
Edited Date: 2015-10-02 06:29 am (UTC)

Date: 2015-10-02 10:03 am (UTC)
From: [identity profile] metaclass.livejournal.com
If we have sufficiently strong type system and adequate language we can represent lazy reference as eight byte long id (and still has 2-3 lower bits for some flags, due to alignment).
But what we have are layers and layers of performance-killing wrappers like in (N)Hibernate and it`s much easier to work with simple types (long,int,whatever) in record fields.

Also these wrappers are not actually transparent. Simple long id cannot cause a LazyInitializationException, unlike some wrapper with reference to long-closed Session.

Date: 2015-10-02 02:42 pm (UTC)
From: [identity profile] juan-gandhi.livejournal.com
Weird idea to issue 10k queries for retrieving 10k records.

(no subject)

From: [identity profile] yatur.livejournal.com - Date: 2015-10-02 02:56 pm (UTC) - Expand

(no subject)

From: [identity profile] juan-gandhi.livejournal.com - Date: 2015-10-02 03:07 pm (UTC) - Expand

Date: 2015-10-02 12:13 pm (UTC)
From: [identity profile] microcell.livejournal.com
You actually never pass real string to the function. Instead what is happening, you are creating new string object in a heap and pass its pointer to the function through the stack. So in a way there is no much difference, but will be less work for a garbage collector.

If all your strings are static and already loaded in some array, the ID will save you trouble to load them each time. Or, if you have some resource provider, like Android does, it will let you to have multi-language support with no cost at all.
Edited Date: 2015-10-02 12:14 pm (UTC)

Date: 2015-10-02 02:40 pm (UTC)
From: [identity profile] juan-gandhi.livejournal.com
I don't know what you are talking about. Creating a new string object in a heap? In JVM? To pass a string to a function? Never heard of this.

(no subject)

From: [identity profile] sassa-nf.livejournal.com - Date: 2015-10-02 04:49 pm (UTC) - Expand

(no subject)

From: [identity profile] microcell.livejournal.com - Date: 2015-10-03 01:26 am (UTC) - Expand

(no subject)

From: [identity profile] juan-gandhi.livejournal.com - Date: 2015-10-03 03:45 am (UTC) - Expand

(no subject)

From: [identity profile] microcell.livejournal.com - Date: 2015-10-03 05:52 am (UTC) - Expand

Date: 2015-10-02 12:29 pm (UTC)
From: [identity profile] sorhed.livejournal.com
Разумеется, это самоочевидная истина. У явных id есть очень немного разумных применений, одно из них такое: звонит, допустим, клиент в саппорт и спрашивает, а что у меня ордер выполнился на два пункта дороже чем надо? А ему в ответ: ща посмотрим, а скажите-ка ID?

(из чего следует, что id для этих целей должен быть красивым и человекочитаемым, например, 137-234-987-202. И вообще генерироваться отдельно, не совпадать с внутренним id, который, как правило, UUIDv4, коих should be enough for everyone).

Ну и ещё из джаваскрипта в скалу приходится id явно гонять, референс же полностью не воссоздать. А так, внутри скального кода, конечно, никаких явных id, кашрут не велит.

Date: 2015-10-02 02:39 pm (UTC)
From: [identity profile] juan-gandhi.livejournal.com
Я на самом деле описал три слоя.

1. id вместо сообщения об ошибке (содержащего конкретные детали)
2. перепасовка целых чисел в коде, вместо type-safe конкретных указателей на объекты
3. использование целых чисел для идентификации явлений природы

По третьему вопросу - конечно, нужен способ; но почему именно десятичные цифры, я не понимаю. base36 для меня звучало бы более осмысленно. BC-4KSNL, то ли дело. Запоминается мгновенно, как мем.

(no subject)

From: [identity profile] microcell.livejournal.com - Date: 2015-10-03 01:56 am (UTC) - Expand

id vs pointer

Date: 2015-10-02 01:36 pm (UTC)
From: [identity profile] a r (from livejournal.com)
1. Id itself is immutable
2. It does not have to live on the same cpu.
3. And there are no life time issues - stale ids are trivially detected.

Edited Date: 2015-10-02 01:37 pm (UTC)

Re: id vs pointer

Date: 2015-10-02 04:51 pm (UTC)
From: [identity profile] sassa-nf.livejournal.com
1. someone needs to maintain a map for Id to memory address
3. tell me about it

Date: 2015-10-04 09:17 pm (UTC)
From: [identity profile] 109.livejournal.com
с содроганием прочитал этот тред. особенно про преимущества int перед long.

> In short. It's stupid to pass around "ids" in a program.

не очень осмотрительно представлять свой очень специфический use case как "всегда" и "везде". это очевидный bias, но надо же себя заставлять.

вообще меня удивляет, как ты дожил до преклонных лет, а до сих пор не знаешь, что правильный ответ на любой вопрос - it depends.

например, я в основном работаю с данными, которые не влезают в память одной машины любого осмысленного размера. мысли создавать врапперы над числовыми id в такой обстановке даже зародиться неоткуда.
Edited Date: 2015-10-05 01:54 am (UTC)

Profile

juan_gandhi: (Default)
Juan-Carlos Gandhi

June 2025

S M T W T F S
1 2345 6 7
8 9 10 11 121314
15161718 1920 21
22232425262728
2930     

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jun. 27th, 2025 06:39 pm
Powered by Dreamwidth Studios